Article ID: 254728 - View products that this article applies to.
This article was previously published under Q254728
IMPORTANT: This article contains information about modifying the registry. Before you modify the registry, make sure to back it up and make sure that you understand how to restore the registry if a problem occurs. For information about how to back up, restore, and edit the registry, click the following article number to view the article in the Microsoft Knowledge Base:
(http://support.microsoft.com/kb/256986/EN-US/ )Description of the Microsoft Windows Registry
The IP Security Protocol (IPSec) does not secure Kerberos or RSVP traffic between Windows 2000 domain controllers, even when the IPSec policy filter is configured to match all IP traffic between the two IP addresses. IPSec, when it is configured to secure all traffic between domain controllers, still lets Kerberos traffic appear as Kerberos traffic on the wire, unprotected by IPSec.
Some types of traffic are exempted by design from being secured by IPSec, even when the IPSec policy specifies that all IP traffic should be secured. The IPSec exemptions apply to Broadcast, Multicast, RSVP, IKE, and Kerberos traffic. Kerberos is a security protocol itself, can be used by IPSec for IKE authentication, and so was not originally designed to be secured by IPSec. Therefore, it is exempt from IPSec filtering. For details about these exemptions, please refer to the following Microsoft Knowledge Base article:
(http://support.microsoft.com/kb/254949/EN-US/ )Client-to-Domain Controller and Domain Controller-to-Domain Controller IPSec Support
To resolve this problem, obtain the latest service pack for Windows 2000. For additional information, click the following article number to view the article in the Microsoft Knowledge Base:
260910No operational affect on Kerberos was observed during internal testing when this fix was applied and Kerberos was IPSec-protected. If you notice problems with Kerberos traffic being secured, but domain trusts not working as a result, please contact Microsoft Product Support Services.
(http://support.microsoft.com/kb/260910/EN-US/ )How to Obtain the Latest Windows 2000 Service Pack
The purpose of this fix is to enable full IPSec protection of all Unicast traffic between two domain controllers, which can now include the Kerberos traffic with the registry key set. This fix is intended to be used on each domain controller, not on Windows 2000 Professional clients. The information in the following Microsoft Knowledge Base article still applies:
254949You should use this fix in conjunction with firewall rules that allow only IPSec and IKE traffic through. For additional information, click the article number below to view the article in the Microsoft Knowledge Base:
(http://support.microsoft.com/kb/254949/ )IPSec support for client-to-domain controller traffic and domain controller-to-domain controller traffic
233256During the boot process, a few packets may be sent before the IPSec driver is initialized and has the IPSec policy fully processed. A properly configured firewall that allows only IKE and IPSec protocols can prevent this non-IPSec traffic from going to inappropriate networks.
(http://support.microsoft.com/kb/233256/EN-US/ )How to Enable IPSec Traffic Through a Firewall
To enable a server to be promoted as a child of a remote domain, set a local IPSec policy that uses certificate authentication. Test the IPSec security association by using Ipsecmon.exe and Ping, or some other method of generating traffic to the remote domain controller. If the IPSec security association is successfully established, all traffic to the remote domain should be protected. The server should be able to join the remote domain, and Dcpromo, Kerberos cross-domain trusts, and normal RPC-based directory replication should all work.
After you apply this hotfix and add the following registry key, you can control the exempt behavior for RSVP and Kerberos with IPSec filter rules. Setting this new registry entry to 1 causes these protocols to be filtered.
WARNING: If you use Registry Editor incorrectly, you may cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that you can solve problems that result from using Registry Editor incorrectly. Use Registry Editor at your own risk.
To add the registry key:
Microsoft has confirmed that this is a problem in Microsoft Windows 2000. This problem was first corrected in Windows 2000 Service Pack 1.
For additional information about how to install Windows 2000 and Windows 2000 hotfixes at the same time, click the article number below to view the article in the Microsoft Knowledge Base:
249149You may have to install a hotfix that is in Windows 2000 SP3 to use Kerberos filtering over IPSec. For additional information, click the article number below to view the article in the Microsoft Knowledge Base:
(http://support.microsoft.com/kb/249149/EN-US/ )Installing Microsoft Windows 2000 and Windows 2000 Hotfixes
(http://support.microsoft.com/kb/309304/EN-US/ )IP Security Transport Mode with Encryption May Drop Fragmented Packets