Select the product you need help with
- Internet Explorer
- Windows Phone
- More products
How to Configure IPSec on an Exchange Server 2003 Back-End Server That Is Running on a Windows Server 2003 Server Cluster
Article ID: 821839 - View products that this article applies to.
IMPORTANT: This article contains information about modifying the registry. Before you modify the registry, make sure to back it up and make sure that you understand how to restore the registry if a problem occurs. For information about how to back up, restore, and edit the registry, click the following article number to view the article in the Microsoft Knowledge Base:
(http://support.microsoft.com/kb/256986/EN-US/ )Description of the Microsoft Windows Registry
This article discusses the use of Internet Protocol security (IPSec) on Exchange 2003 back-end servers that are running on a Windows Server 2003-based server cluster.
Microsoft supports Exchange 2003 running on Windows Server 2003-based computers and using IPSec transport mode Encapsulating Security Payload (ESP) to encrypt communication with clustered Exchange 2003 servers that use Network Load Balancing clusters or server clusters. When you use IPSec between front-end servers and back-end servers, failover times depend on Exchange 2003 recovery time plus the time it takes for IPSec to renegotiate security associations during the failover process.
Front-end Exchange 2003 servers such as Outlook Web Access (OWA) servers cannot use Secure Sockets Layer (SSL) or Transport Layer Security (TLS) to encrypt traffic to back-end Exchange 2003 servers. The communication between front-end servers and back-end servers always uses unencrypted forms of traffic when Hypertext Transfer Protocol (HTTP), Post Office Protocol version 3 (POP3), or Internet Messaging Access Protocol 4 (IMAP4) are used.
In Exchange 2003, the HTTP protocol uses secure authentication where possible. The Exchange 2003 front-end server uses Kerberos or NTLM authentication to communicate with the back-end server if the back-end server is configured to accept integrated Windows authentication. This helps protect user password information from a malicious user who might try to capture the network traffic between the front-end server and the back-end server.
The POP3 protocol and the IMAP4 protocol can only use basic authentication to communicate with a back-end server. Because of this limitation, user password information is not protected against a malicious user who might try to capture network traffic between the front-end server and the back-end server.
The following list describes some of the reasons why you may want to use IPSec to establish trust and to encrypt network traffic between an Exchange 2003 front-end server and a back-end server:
(http://support.microsoft.com/kb/306677/EN-US/ )IPSec Is Not Designed for Failover
248346When you configure IPSec on a back-end server in a Windows Server 2003 server cluster, the following behavior occurs when you use the Cluster Administrator utility to move a clustered resource that uses a virtual IP address:
(http://support.microsoft.com/kb/248346/ )L2TP Sessions Lost When Adding a Server to an NLB Cluster
In a scenario where the Microsoft Cluster service back-end cluster node stops responding (crashes), the Cluster service starts the failover procedure for the clustered program and the virtual IP address . However, in this case, the front-end IPSec computer still believes it has a secure link to the virtual IP address. The IPSec component uses its idle timer to determine that the back-end node no longer exists. On a Windows 2000-based computer, the idle time minimum is 5 minutes. On a Windows Server 2003-based computer, the idle time is automatically reduced to 1 minute if the
NLBSFlagsregistry key is set. As soon as IPSec removes the idle IPSec security associations, IKE tries to renegotiation new IPSec security associations. After 1 minute, IKE tries to establish a new Main Mode negotiation to the virtual IP address and is then successful in creating new security associations with the new cluster node. Because of this procedure, the total time it takes for IPSec to fail over is 6 minutes: the IPSec idle time of 5 minutes plus 1 minute for IKE to renegotiate a new Main Mode negotiation with the virtual IP address.
To Modify the Renegotiation TimeTo allow IPSec to renegotiate the session to a back-end cluster node in a shorter period of time after an unexpected failover, set the
NLBSFlagsregistry value in the following registry subkey on the front-end Exchange 2003 server:
To do so:
WARNING: If you use Registry Editor incorrectly, you may cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that you can solve problems that result from using Registry Editor incorrectly. Use Registry Editor at your own risk.
Suggested IPSec Policy DesignWhile this article does not provide step-by-step instructions to configure IPSec policies, the following suggested IPSec policy implementations may be used with Exchange 2003:
Article ID: 821839 - Last Review: October 25, 2007 - Revision: 1.4