Select the product you need help with
- Internet Explorer
- Windows Phone
- More products
IPsec troubleshooting in Microsoft Windows 2000 Server
Article ID: 257225 - View products that this article applies to.
This article was previously published under Q257225
To troubleshoot IPsec connection problems in Microsoft Windows 2000, first verify the success of Internet Key Exchange (IKE) security negotiation. To do this, enable Audit policy and then examine the Security log. Next, use the Netdiag.exe command-line tool to display debugging information. Then, depending on whether the problem occurs in phase one or in phase two, examine your IPsec policy properties and IPsec rules.
Use IP Security Monitor to view more information about IPsec and security associations. You can also use IP Security Monitor to view IKE statistics. Use Network Monitor to analyze network traffic and the status of the various protocols used in your network. You can use the Netsh command to troubleshoot instances where IP offloading occurs on IPsec packets.
You can also use the information in this article to do the following:
This article contains guidelines for troubleshooting Internet Protocol security (IPsec) connection problems in Microsoft Windows 2000. IPsec relies on the Internet Key Exchange (IKE) protocol to establish shared security parameters and authenticated keys between two computers. The IKE protocol uses two phases. In phase one, Windows 2000 uses the IKE Security Association and Key Management Protocol (ISAKMP) Main Mode exchange. (Windows 2000 does not support Aggressive Mode.) When the phase one exchange provides a secured channel, the computers obtain an authenticated key and an IKE security association. This secured channel is used in phase two to help protect the Quick Mode Exchange. The Quick Mode Exchange provides IPsec security associations.
Basic IPsec troubleshootingTo troubleshoot IPsec, first enable Audit policy, and then verify the results of phase one and phase two exchanges. When you enable Audit policy, security events are logged in the Security log. By examining the Security log, you can determine whether IKE security association negotiation is successful. To enable Audit policy, follow these steps:
Next, type the following command to use the Netdiag.exe command-line tool:
netdiag /test:ipsec /debugThis command displays debugging information about phase two.
Note To use Netdiag.exe, the Windows 2000 Support Tools package must be installed on your computer. To install the Windows 2000 Support Tools, follow these steps:
netdiag /test:ipsec /vThis command displays the current policy and IPsec statistics with regard to phase one.
If the logged events indicate that phase one Main Mode exchange fails, verify the IKE settings and the IKE authentication methods in your IPsec policy properties. To do this, follow these steps:
Using IP Security MonitorYou can use IP Security Monitor to monitor your security associations, IPsec statistics, and IKE statistics. In particular, you can use IP Security Monitor to verify the success of authentication and security associations. To start IP Security Monitor, click Start, click Run, type ipsecmon, and then click OK.
Note By default, IP Security Monitor displays statistics for the local computer. To specify a remote computer, click Start, click Run, type ipsecmon computer_name, and then click OK.
The upper group box in the IP Security Monitor dialog box displays the active security associations and the configuration of the active policy. The lower left group box displays the following IPsec statistics:
Using Network MonitorYou can use Network Monitor to analyze the following:
Obtaining an Oakley logImportant This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:
(http://support.microsoft.com/kb/322756/ )How to back up and restore the registry in Windows
Developers and network administrators who have advanced IKE knowledge can modify the registry to obtain an Oakley log. To do this, use Registry Editor to locate the following registry subkey. If the subkey does not exist, create it.
Add an entry of the REG_DWORD type value named "EnableLogging." Give this entry a value of 1. When this entry takes effect, an Oakley.log file is created in the %systemroot%\Debug folder.
Note To turn off logging, give the EnableLogging entry a value of 0.
Using the Netsh commandYou can use the Netsh command to troubleshoot instances where IP offloading occurs on IPsec packets. IP offloading occurs when the network card instead of the CPU performs IP functions. For example, IP offloading occurs when the network card performs checksum calculations or performs packet encryption and decryption. IP offloading causes the IPsec driver to drop the packet. To determine whether an interface can perform IP offloading, follow these steps:
To disable IP offloading, follow these steps to modify the registry:
Event logsThe following events may be logged in the Security event log:
Troubleshooting "bad SPI" messages in the Event Viewer"Bad SPI" messages are logged in the following circumstances:
Configuring longer values may not prevent bad SPIs. However, configuring longer values can significantly reduce the number of bad SPIs. Typically, Windows 2000 Server logs event 4268 to indicate that packets were discarded because of a bad SPI.
If IP Security Monitor indicates that secured security associations are not established, nonsecure security associations may be preventing secured security associations from being established.
Note A secured security association is also known as a hard security association. An nonsecure security association is also known as a soft security association.
Run IP Security Monitor on one of the peer computers. If a security association exists, and the security setting is None, an nonsecure security association exists. An nonsecure security association remains on the computer as long as traffic is regularly sent. To prevent this condition, stop all traffic until the security association times out. Typically, the security association times out in five minutes. Use IP Security Monitor to make sure that the security association is no longer established, and then start traffic again. If policies are compatible, a secured security association is automatically established. Restart the policy agent to delete all nonsecure security associations.
If the files that are required for IPsec components have been removed or deleted, reinstall the IPsec components by removing and then reinstalling the TCP/IP network protocol. Files that IPsec components require include the following:
IPsec negotiations may fail because of incompatible IPsec policy settings. Examine the Security event log on each computer that participates in a negotiation. Recent events may record attempts to perform an Oakley negotiation. The events may include a description of the success or the failure.
Verify the integrity of the policy on each computer. To determine the cause of a policy mismatch, follow these steps:
Restarting the policy agentWhen you restart the policy agent, you remove old or nonsecure security associations. Restart the policy agent if IP Security Monitor does not show any security negotiations. Also restart the policy agent if you want to download a policy from the domain or from the policy store.
Verifying policy integrityActive Directory assumes that the most recent changes are current. However, if multiple administrators try to change a policy at the same time, the links between policy components may break. A policy integrity check resolves this problem by verifying the links in all IPsec policies. Run an integrity check after any modifications are made to a policy. To test IPsec policy integrity, follow these steps:
Reviewing the IPsec driver and policy agent registry settingsThe settings for the IPsec driver are located in the following registry subkey:
You can modify the values of the following entries:
You can modify the values of the following entries:
231585For more information about Layer 2 Tunneling Protocol (L2TP)/IPsec connections, click the following article number to view the article in the Microsoft Knowledge Base:
(http://support.microsoft.com/kb/231585/ )Overview of secure IP communication with IPsec in Windows 2000
(http://support.microsoft.com/kb/248750/ )Description of the IPSec policy created for L2TP/IPSec
Article ID: 257225 - Last Review: October 12, 2007 - Revision: 7.3