Troubleshooting long running MAPI connections to Exchange Server 2010 through Network Load Balancers

Symptoms

Consider the scenario where you have Outlook or other MAPI applications that connect to Exchange Server 2010 through a hardware load balancer or other network device. The application also requires long running TCP connections to function as expected. If the network device closes idle connections via a timeout setting, the MAPI application fails to complete requests over the connection. 

If the application is Outlook, you may see the following events in the Application event log of the client computer:

Log Name: Application
Source: Outlook
Event ID: 25
Task Category: None
Level: Information
Keywords: Classic
User: N/A
Computer: Computer.contoso.com
Description:Could not perform the desired operation. The selected command is not valid for this recipient. The connection to Microsoft Exchange is unavailable. Outlook must be online or connected to complete this action..

Log Name: Application
Source: Outlook
Event ID: 26
Task Category: None
Level: Information
Keywords: Classic
User: N/A
Computer: Computer.contoso.com
Description:Connection to Microsoft Exchange has been restored.

Cause

A configuration for a hardware load balancer, or other network device such as a firewall or proxy, is causing client connections to drop. When the device closes idle connections, via timeout or other configuration, the application fails to complete subsequent requests over the connection that was closed.

Resolution

To resolve this issue, consider the following solutions:

1. Verify the configuration of the network device, per the vendor recommendations or through contacting the Technical Support team for the vendor. This may require increasing the timeout on the network device so that long running connections are not prematurely closed.

2. For an application server connecting to Exchange Server 2010, set the application server's TCP Keep-Alive time to a value less than the idle connection timeout window on the network device. 
   
   For example, if the idle connection time out on the device is 15 minutes, set the TCP Keep-Alive on the application server to a value less than 15 minutes.

By default the TCP Keep-Alive time is 2 hours and disabled. To adjust the TCP Keep-Alive time on an application server see the following article from the Microsoft Knowledge Base:

324270 How to harden the TCP/IP stack against denial of service attacks in Windows Server 2003It is important to stress that the connections in question are typically internal RPC connections for Outlook clients in Online mode, rather than RPC over HTTP/Outlook Anywhere connection. Frequently, issues such as this occur due to session timeouts, network configuration, optimization configuration or other features on the network device that require modification. 

In many cases, session timeout settings are typically too low. In discussion with network device vendors, it is frequently recommended that RPC timeout settings should be 120 minutes, at a minimum.

More Information

Both of the solutions above may prevent the TCP connection from timing out prematurely. 

Note A frequently used test to determine whether a device configuration is at issue is to bypass the device. This is done by using a Hosts file on the client or a DNS record to access a single CAS server in the array. However, if the device is also used as a gateway, this may not provide accurate results.

For additional information on long running connections, see section 4.2.3.6, TCP Keep-Alives, from the following Internet Engineering Task Force document:

RFC 122 - Requirements for Internet Hosts - Communication Layers
http://datatracker.ietf.org/doc/rfc1122/

For additional information on Load Balancing for Exchange Server 2010, see the following presentation from TechEd 2011

Load Balancing with Microsoft Exchange Server 2010
http://channel9.msdn.com/Events/TechEd/NorthAmerica/2011/EXL307