Bugcheck 0xF4 occurs on a Hyper-V guest VM running in Windows Server 2008 R2 while under stress when MPIO PathVerify is enabled

Applies to: Windows Server 2008 R2 DatacenterWindows Server 2008 R2 EnterpriseWindows Server 2008 R2 Standard

Symptoms


With the following configuration, guest VM systems may experience Stop 0xF4 errors:

·       Windows Server 2008 R2 x64 with Hyper-V Role installed

·       iSCSI and Multi-Path I/O (MPIO) running

·       Guest VM boot storage (VHD files) on iSCSI storage

·       MPIO path verification is enabled

The stop errors in the guest systems are more likely to occur after a short network interruption, such as a failover of an iSCSI target controller, while under heavy load.

In this scenario, this Windows Server 2008 R2-based computer may unexpectedly crash in the Csrss.exe process. Additionally, you may receive the following Stop error:

STOP 0x000000F4 (0x00000003, parameter 2, parameter 3, parameter 4)
CRITICAL_OBJECT_TERMINATION

Cause


There are two conditions that can work together to result in extended iSCSI outages, which may result in the guest system failures.

1. Both the Windows iSCSI and MPIO drivers extensively use worker queue threads to queue work items for processing on iSCSI initiator systems.

Under heavy stress, the delay in worker thread processing may be long enough to result in disk timer expiration on the guest systems. This will cause the removal of the boot drive from the guest system that result in the bugcheck.

2. When PathVerify in MPIO is enabled, it can greatly increase the network and worker queue thread contention, especially if the path verify interval (PathVerificationPeriod) is short.

Resolution


Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:

 322756How to back up and restore the registry in Windows

Note: All of the registry changes in this section require a reboot to take effect.

This problem can be worked around by configuring Windows registry settings to increase the number of worker queue threads, and increase the delay between path verification commands.

Steps to modify the worker thread registry value:

1. Click Start, type regedit in the Start Search box, and then click regedit in the Programs list.

If you are prompted for an administrator password or confirmation, type your password or click Continue.

2. Locate and then click the following registry subkey:

·         HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Executive

3. In the details pane, right-click AdditionalCriticalWorkerThreads , and then click Modify.

4. In the Value data box, type 10 (with Base set to Hexadecimal, or 16 in Decimal), and then click OK.

5. Right-click AdditionalDelayedWorkerThreads , and then click Modify .

6. In the Value data box, type 10 (with Base set to Hexadecimal, or 16 for Decimal), and then click OK.

Note: You can specify a value that is between 0 and the maximum value of 0x64 (hexadecimal), or 100 (decimal), for both of these fields. If a larger value is specified, the maximum value will be used.

Exit Registry Editor.

Steps to modify the path verification registry values:

The PathVerificationPeriod is only relevant if path verification is enabled. To verify whether or not path verification is enabled on your system, see the following registry entry:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MPIO\Parameters\
Setting Definition
PathVerifyEnabledFlag that enables path verification by MPIO on all paths every N seconds (where N depends on the value set in PathVerificationPeriod).

Type is boolean and must be filled with either 0 (disable) or 1 (enable). By default, it is disabled.
 

1. Click Start, type regedit in the Start Search box, and then click regedit in the Programs list.

If you are prompted for an administrator password or confirmation, type your password or click Continue.

2. Locate and then click the following registry subkey:

·         HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\mpio\Parameters

3. In the details pane, right-click PathVerificationPeriod, and then click Modify.

4. In the Value data box, type 3C (with Base set to Hexadecimal, or 60 in Decimal), and then click OK.

Exit Registry Editor.

More Information


See the TechNet MPIO timer configuration page for more details.