Forums guru3d com threads windows line based vs message signaled based interrupts msi tool 378044

… or another attempt to improve latencies

Little bit of theory:
*****
From «Windows Internals» by Mark Russinovich, David A. Solomon, Alex Ionescu
Line-Based vs. Message Signaled-Based Interrupts
Shared interrupts are often the cause of high interrupt latency and can also cause stability issues. They are typically undesirable and a side effect of the limited number of physical interrupt lines on a computer. For example, in the previous example of the 7-in-1 media card reader, a much better solution is for each device to have its own interrupt and for one driver to manage the different interrupts knowing which device they came from. However, consuming four IRQ lines for a single device quickly leads to IRQ line exhaustion. Additionally, PCI devices are each connected to only one IRQ line anyway, so the media card reader cannot use more than one IRQ in the first place.

Other problems with generating interrupts through an IRQ line is that incorrect management of the IRQ signal can lead to interrupt storms or other kinds of deadlocks on the machine, because the signal is driven “high” or “low” until the ISR acknowledges it. (Furthermore, the interrupt controller must typically receive an EOI signal as well.) If either of these does not happen due to a bug, the system can end up in an interrupt state forever, further interrupts could be masked away, or both. Finally, line-based interrupts provide poor scalability in multiprocessor environments. In many cases, the hardware has the final decision as to which processor will be interrupted out of the possible set that the Plug and Play manager selected for this interrupt, and there is little device drivers can do.

A solution to all these problems is a new interrupt mechanism first introduced in the PCI 2.2 standard called message-signaled interrupts (MSI). Although it remains an optional component of the standard that is seldom found in client machines, an increasing number of servers and workstations implement MSI support, which is fully supported by the all recent versions of Windows. In the MSI model, a device delivers a message to its driver by writing to a specific memory address. This action causes an interrupt, and Windows then calls the ISR with the message content (value) and the address where the message was delivered. A device can also deliver multiple messages (up to 32) to the memory address, delivering different payloads based on the event.

Because communication is based across a memory value, and because the content is delivered with the interrupt, the need for IRQ lines is removed (making the total system limit of MSIs equal to the number of interrupt vectors, not IRQ lines), as is the need for a driver ISR to query the device for data related to the interrupt, decreasing latency. Due to the large number of device interrupts available through this model, this effectively nullifies any benefit of sharing interrupts, decreasing latency further by directly delivering the interrupt data to the concerned ISR.

Finally, MSI-X, an extension to the MSI model, which is introduced in PCI 3.0, adds support for 32-bit messages (instead of 16-bit), a maximum of 2048 different messages (instead of just 32), and more importantly, the ability to use a different address (which can be dynamically determined) for each of the MSI payloads. Using a different address allows the MSI payload to be written to a different physical address range that belongs to a different processor, or a different set of target processors, effectively enabling nonuniform memory access (NUMA)-aware interrupt delivery by sending the interrupt to the processor that initiated the related device request. This improves latency and scalability by monitoring both load and closest NUMA node during interrupt completion.
*****

Now practice.
Checking for PCI devices working in MSI-mode.
Go to Device Manager. Click in menu «View -> Resources by type». Expand «Interrupt request (IRQ)» node of the tree. Scroll down to «(PCI) 0x… (…) device name» device nodes. Devices with positive number for IRQ (like «(PCI) 0x00000011 (17) …») are in Line-based interrupts-mode. Devices with negative number for IRQ (like «(PCI) 0xFFFFFFFA (-6) …») are in Message Signaled-based Interrupts-mode.

Trying to switch device to MSI-mode manually.
You must locate device`s registry key. Invoke device properties dialog. Switch to «Details» tab. Select «Device Instance Path» in «Property» combo-box. Write down «Value» (for example «PCI\VEN_1002&DEV_4397&SUBSYS_1609103C&REV_00\3&11583659&0&B0»). This is relative registry path under the key «HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\».

Go to that device`s registry key («HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\PCI\VEN_1002&DEV_4397&SUBSYS_1609103C&REV_00\3&11583659&0&B0») and locate down the subkey «Device Parameters\Interrupt Management». For devices working in MSI-mode there will be subkey «Device Parameters\Interrupt Management\MessageSignaledInterruptProperties» and in that subkey there will be DWORD value «MSISupported» equals to «0x00000001». To switch device from legacy- to MSI-mode just add these subkey and value.

Before adding these key and value (or changing «MSISupported» to «0x00000001» in case subkey and value already exist) you have to perform safety steps like doing backup (creating system restore point at least).

Do tweak one device -> reboot and check (1) if it is displayed in Device Manager as correctly working device; (2) if its IRQ became negative -> if no (1) and no (2) then either remove subkey «MessageSignaledInterruptProperties» (if you added it) or change «MSISupported» to «0x00000000» and reboot.

Theoretically if device driver (and platform = chipset) unable to perform in MSI-mode it should ignore mentioned subkey and value. But who knows…

Trying to switch device to MSI-mode with MSI utility v3 (run as administrator)
http://www.mediafire.com/file/ewpy1p0rr132thk/MSI_util_v3.zip/file
MD5 hash for zip-file: 8424509737CEDBDE4BA9E9A780D5CE96

Change log:
— showing multiple IRQs for device (previous version showed only first of multiple IRQs);
— extending the details pane (under the grid) for the selected device to show more properties — PNP and PCI ones;
— replacing WMI with Setup API for retrieving device properties — so all info should be available on Win7/Win8 rigs;
— replacing registry names of devices with normal PNP display names from Setup API;
— adding the ability to launch the registry editor with selected device instance by double click on the device.

Edit: If you have old rig and there are no devices working in MSI-mode, then may be you should not experiment with this tweak.
As this tweak is risky then you probably should not tweak devices which can prevent Windows to boot (by their failure) — SATA controllers, videocards. Especially if they do not share their IRQ with another devices.

Edit: Troubleshooting topic with info from «Windows Internals»:

Besides starting services, the system charges the Sevice Control Manager (SCM) with determining when the system’s registry configuration, HKLM\SYSTEM\CurrentControlSet, should be saved as the last known good control set. The CurrentControlSet key contains the Services key as a subkey, so CurrentControlSet includes the registry representation of the SCM database. It also contains the Control key, which stores many kernel-mode and user-mode subsystem configuration settings. By default, a successful boot consists of a successful startup of auto-start services and a successful user logon. A boot fails if the system halts because a device driver crashes the system during the boot or if an auto-start service with an ErrorControl value of SERVICE_ERROR_SEVERE or SERVICE_ERROR_CRITICAL reports a startup error.

The SCM obviously knows when it has completed a successful startup of the auto-start services, but Winlogon (%SystemRoot%\System32\Winlogon.exe) must notify it when there is a successful logon. Winlogon invokes the NotifyBootConfigStatus function when a user logs on, and NotifyBootConfigStatus sends a message to the SCM. Following the successful start of the auto-start services or the receipt of the message from NotifyBootConfigStatus (whichever comes last), the SCM calls the system function NtInitializeRegistry to save the current registry startup configuration.

Windows maintains several copies of CurrentControlSet, and CurrentControlSet is really a symbolic registry link that points to one of the copies. The control sets have names in the form HKLM\SYSTEM\ControlSetnnn, where nnn is a number such as 001 or 002. The HKLM\SYSTEM\Select key contains values that identify the role of each control set. For example, if CurrentControlSet points to ControlSet001, the Current value under Select has a value of 1. The LastKnownGood value under Select contains the number of the last known good control set, which is the control set last used to boot successfully. Another value that might be on your system under the Select key is Failed, which points to the last control set for which the boot was deemed unsuccessful and aborted in favor of an attempt at booting with the last known good control set. Figure 4-15 displays a system’s control sets and Select values.

NtInitializeRegistry takes the contents of the last known good control set and synchronizes it with that of the CurrentControlSet key’s tree. If this was the system’s first successful boot, the last known good won’t exist and the system will create a new control set for it. If the last known good tree exists, the system simply updates it with differences between it and CurrentControlSet. Last known good is helpful in situations in which a change to CurrentControlSet, such as the modification of a system performance-tuning value under HKLM\SYSTEM\Control or the addition of a service or device driver, causes the subsequent boot to fail. Users can press F8 early in the boot process to bring up a menu that lets them direct the boot to use the last known good control set, rolling the system’s registry configuration back to the way it was the last time the system booted successfully.

So if you choose to try and switch PCI device into the message-mode, you better do add «MSISupproted» registry value to one device and reboot. If Windows failed to boot you restart and press F8 and choose boot option «last known good configuration» (which must be the configuration without «MSISupported»).

Edit: Improper Propagation of SATA Message Signaled Interrupts on AMD SB950

Description
The SATA Message Signaled Interrupt (MSI) from higher port numbers may not properly propagate to port 0 before being sent to the driver.

Potential Effect on System
On a platform with SATA MSI enabled and activity occurring on SATA devices connected to three or more ports, if the failing condition occurs, the driver will not receive the interrupt and the system will hang.

Suggested Workaround
Do not expose the MSI capability in the SATA controller. This change is implemented in CIMx version 1.1.0.2.

Fix Planned
No

Edit: Here is Intel`s paper (thanks to n1hilist):
http://www.intel.co.za/content/dam/…hite-papers/msg-signaled-interrupts-paper.pdf

Edit: Easy way to measure ISRs and DPCs
https://forums.guru3d.com/threads/simple-way-to-trace-dpcs-and-isrs.423884/

Edit: Dedicated thread about interrupts in Windows
https://forums.guru3d.com/threads/a-bit-detailed-info-about-interrupts-in-windows.424677/

Edit: External Interrupts in the x86 system. Part 1. Interrupt controller evolution
https://habr.com/en/post/446312/

Edit: Use this site to check hash value for files in this post — https://hash.online-convert.com/md5-generator
Even if it gives wrong hashes using the same site proves the un-compromised state.

It`s my clumsy english. Sorry.

As for MSI-mode, device itself has to use (implement) MSI capability, and its driver too.

And what about network card and SATA controller in your rig? I suspect that video card is not the source of mass interrupt bursts…

I have slowly got into Von’s page on performance tweaks. Many i’ve done before and or found else where so i found myself either repeating or skipping.

I monkeyed with tonight with the IRQ and upon reboot i got a blue screened before windows could log in. I ended up just reinstalling windows because i did not have system restore on that partition as i do with this one; why i dont remember.

And with a new fresh install i can hopefully nail what is causing my problems by introducing 1 thing at a time.

I did core parking and the network throttling i put to 0.

This maybe unheard off but by god i believe its keyboard or mouse related but there is no way to really test.No way mean i dont have replacements or software that i know of to find problems. I have g100 keyboard and g500 mouse.
It seems that my disconnects only happen during me actually using my hardware. Example me spectating in call of duty or bf3 or tf2i have yet to see this same phenomenon happen. My disconnects and by this i mean my game freezes and connection drops and returns are made worse by excessive input from me. So putting the peices together makes me rule out internet issues however it acts just like an internet issue. Hell it reminds of the House episode where they believe its cancer but every test shows that is not cancer. Strange.

I have no idea gentlemen. I am ready to slap this on ebay, all of it, and say the hell with it. I’ll build from the ground up when i have time.

In vons thread he goes into device manager to disable stuff, so i did to. Looking under device manager i see x4 HID complaint keyboard and same for mouse. I see «virtual keyboard» and «virtual mouse». Which makes me wonder, but that about it.

I dont know. I would seriously pay someone to sit at my computer and do all that is necessary to find out why i can’t use it.

I`ve looked at the picture and can say, that at least your video-cards and SATA controller do not share IRQ with another devices. But your Marvell Yukon NIC does share IRQ (18) with USB controllers 3A44 and 3A3C. In case these particular USB controllers have some USB devices attached to them you better connect USB devices to another ports to try to avoid those USB controllers usage. If no devices connected to them USB controllers you can disable them in Device Manager to provide Marvell NIC with exclusive IRQ.

And line-mode of devices is no reason to bang the wall. It is normal mode but merely less optimal then message-mode.

Well, beeing a programmer in the company whose main product line is software for hard disk management, backup and restore, I just use bootable DVD with WinPE and our software to backup internal HDD to the external HDD. Imo this is the best protection against any possible damage.

But I was sure that if you fail to boot you always have boot option «last known good configuration»….

And once again — better try one device per attempt (attempt per device?).

Isn’t WinPE a lot like the recovery console?
Problem is, I think the CurrentControlSet key is volatile, because when I used my method it did not exist. I had to change (aka, delete) the MessageSignaledInterruptProperties keys from the ControlSet001, ControlSet002, and ControlSet003 keys instead.

Doesn’t «Last Known Good Configuration» just take you back to your most recent windows recovery backup that booted? I mean sure that would work, but even that is more annoying than using the recovery console.

Yeah, when I build up the nerve I’ll give it another go another time.

Not a console: http://en.wikipedia.org/wiki/Windows_Preinstallation_Environment . I mean it has GUI and all that.

As for CurrentControlSet — it is namely a link to the one last known good key of ControlSetNNN keys.

Edit:

Without «recovery backup» words. Every time Windows successfully started it creates a copy of ControlSetNNN key. And if you choose that boot option Wondows must discard its newest ControlSetNNN key and take previous one. Imo it is simple and quick.

Yeah WinPE is very similar. Modern day recovery tool is pretty much a preinstallation environment.
http://partition-recovery.com/images/Vista_Repair_2.jpg
I used that last option there, and used regedit from cmd, gui works fine. Problem is, it loads its own registry hives. So I needed to load the system hive, make changes, and then unload it. Very annoying but at least it works, and I’m guessing since CurrentControlSet -I think- is made on startup it did not exist for me. Dose WinPE load the registry directly from your windows installation?

Because like I said, the key did not exist for me outside of windows, but the changes I made to the CurrentControlSet key while in windows must have been applied to the entries of the devices in the ControlSet001-3… if that makes sense…

Point is, it’s annoying lol.

Regarding Last Known Good Configuration, I believe you. But for whatever reason I did not get that option. It wanted me to scan the whole windows installation for errors that could prevent startup (and thus ruining all the customization and visual tweaks I have).

No. And I don`t use WinPE to edit registry. In that you are more advanced user.
Out of my pedantry and tediousness — CurrentControlSet is not real key, it is link to one of CurrentControlSetNNN keys.

Edit:

Sounds like you are talking about recovery console stage. That «Last known» feature is at the previous stage of «F8» boot menu. I don`t know about Win8 though.

Oh, alright, explains why the key did not show up.
And I’m just lucky I had my girlfriends laptop to google how to load and unload registry hives from the recovery console. (without a disc).

edit: maybe that is what I am thinking. I remember Last Known Config showing up after a unsuccessful boot in the past, so I assumed that would not work for me and used F8 and chose «repair your computer» (which does not really scan anything, it’s that recovery console picture I posted).

Alright, at least I have more options when I go at it again. Though, this kinda makes backing up the registry redundant.

Probably I should copy here a piece of «Windows internals» about that «Last known good configuration» feature intrinsics. For a troubleshoot info.

Edit: Done… Read at the OP.

You can change IRQ only by switching device to MSI-mode. I don`t know other ways. In old Windows user could change IRQ in device properties. But now all controls are disabled there. So if you can`t switch device to MSI-mode but have devices sharing the IRQ you can disable not used devices to avoid such sharing.