On a Microsoft Windows Server 2008 R2 Fail over cluster with a Hyper-V guest with many pass-through disks, the machine configuration may take some time to come online

Consider the following Scenario: 

• You have a Windows Server 2008 R2 failover cluster with Hyper-V
• You have at least one Hyper-V guest added to this cluster
• You have many pass-through disks added to this guest that are being managed by the cluster
• You have multiple paths to storage over iSCSI and are using an active/active policy in MPIO like Round Robin

The following actions via Failover Cluster Manager take an extended period of time to complete or fails (times out):

• Bringing the Virtual Machine configuration online.
• Failing over the Virtual Machine
• Refreshing the Virtual Machine configuration in Failover Cluster Manager
• Automatic refresh of the Virtual Machine configuration after a configuration change

The more pass-through disks you have presented to a particular Virtual Machine, the longer it will take for its Virtual Machine configuration to online or refresh.

The time taken is proportionate to the amount of disks configured in that Virtual Machine.  This online time will extend until it fails because it exceeds the default resource pending or deadlock time out for the Virtual Machine configuration cluster resource type.

Slow iSCSI performance may occur during network congestion or when RFC 1122-delayed acknowledgements extend the error recovery process.  In situations where there is no network congestion, the default 200 millisecond delay on the acknowledgement can significantly impact SCSI command latency in an iSCSI environment.  Use of multipathing solutions which load balancing read requests across multiple array ports, increases the likelihood that those SCSI OpCode read requests (Read Lun, ReadLunCapacity) completions from multiple ports will result in network congestion. 

The Nagle Algorithm applied as part of RFC 1122 means that small payload network transmissions (such as read SCSI OpCode commands) may not be sent until either a full TCP segment is reached on that NIC or the delayed acknowledge time trigger is reached (200ms).

The online operation of a Virtual Machine Configuration performs a sanity check to ensure that all VM components, including storage are available and accessible.  In this occurrence, there are multiple SCSI read commands which need to occur, serially per pass-through disk.  Due to the delayed acknowledgement, each one of these commands take up to 200ms to complete, therefore extending the overall time to online this VM configuration resource.

Typically, an acknowledgment is sent for every other TCP segment that is received on a connection unless the delayed ACK timer (200 milliseconds) expires. You can adjust the delayed ACK timer by editing the registry as outlined in the workaround below.

Modify the TCP/IP settings for the network interfaces carrying iSCSI traffic to immediately acknowledge incoming TCP segments.  This workaround solves the read performance issue.    

Note: These TCP/IP settings should not be modified for network interfaces not carrying iSCSI traffic as the increased acknowledgement traffic may negatively affect other applications.

Caution! This workaround contains information about modifying the registry. Before you modify the registry, make sure to back it up and make sure that you understand how to restore the registry if a problem occurs. For information about how to back up, restore, and edit the registry, click the following link to view the following article in the Microsoft Knowledge Base:  http://support.microsoft.com/kb/256986/.

1. Start Registry Editor (Regedit.exe).
    
2. Locate and then click the following registry subkey:
   
   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces
   
   the interfaces will be listed underneath by automatically generated GUIDs like {064A622F-850B-4C97-96B3-0F0E99162E56}
    
3. Click each of the interface GUIDs and perform the following steps:
   
   a.  Check the IPAddress or DhcpIPAddress parameters to determine whether the interface is used for iSCSI traffic.  If not, skip to the next interface.
   
   b.  On the Edit menu, point to New, and then click DWORD value.
   
   c.  Name the new value TcpAckFrequency, and assign it a value of 1.
    
4. Exit the Registry Editor.
    
5. Restart Windows for this change to take effect.

You can view further information about this in the iSCSI user guide. 

http://download.microsoft.com/download/a/e/9/ae91dea1-66d9-417c-ade4-92d824b871af/uguide.doc

Note This is a "FAST PUBLISH" article created directly from within the Microsoft support organization. The information contained herein is provided as-is in response to emerging issues. As a result of the speed in making it available, the materials may include typographical errors and may be revised at any time without notice. See Terms of Use (http://go.microsoft.com/fwlink/?LinkId=151500) for other considerations.