To troubleshoot IPsec connection problems in Microsoft Windows 2000, first verify the success of Internet Key Exchange (IKE) security negotiation. To do this, enable Audit policy and then examine the Security log. Next, use the Netdiag.exe command-line tool to display debugging information. Then, depending on whether the problem occurs in phase one or in phase two, examine your IPsec policy properties and IPsec rules.
Use IP Security Monitor to view more information about IPsec and security associations. You can also use IP Security Monitor to view IKE statistics. Use Network Monitor to analyze network traffic and the status of the various protocols used in your network. You can use the Netsh command to troubleshoot instances where IP offloading occurs on IPsec packets.
You can also use the information in this article to do the following:
Basic IPsec troubleshooting
- Start Windows 2000.
Note You must log on as a member of the administrator group to install these tools.
- Insert the Windows 2000 CD into your CD drive.
- Click Browse this CD, and then open the Support\Tools folder.
- Double-click Setup.exe, and then follow the instructions that appear on the screen.
- Click Start, click Run, type secpol.msc, and then click OK.
- Click the IPsec rule that you want to click, right-click IPsec rules and then click Properties.
- Click the General tab, and then verify that the settings are correct.
- Click Advanced, examine the settings, click Methods, and then examine the settings.
- Click OK two times.
- Click Rules tab, click Edit, and then click the Authentication Methods tab.
- Examine the settings on this tab.
Using IP Security Monitor
You can use IP Security Monitor to monitor your security associations, IPsec statistics, and IKE statistics. In particular, you can use IP Security Monitor to verify the success of authentication and security associations. To start IP Security Monitor, click Start, click Run, type ipsecmon, and then click OK.
Note By default, IP Security Monitor displays statistics for the local computer. To specify a remote computer, click Start, click Run, type ipsecmon computer_name , and then click OK.
The upper group box in the IP Security Monitor dialog box displays the active security associations and the configuration of the active policy. The lower left group box displays the following IPsec statistics:
The number of active security associations.
Confidential Bytes Sent
The number of bytes sent by using the Encapsulating Security Payload (ESP) security protocol (decimal ID 50).
Confidential Bytes Received
The number of bytes received by using the ESP security protocol.
Authenticated Bytes Sent
The number of bytes sent with the authentication property enabled.
Authenticated Bytes Received
The number of bytes received with the authentication property enabled.
Bad SPI Packets
The number of packets whose Security Parameters Index (SPI) is not valid. A positive number probably indicates that the security association has expired or is no longer valid.
The SPI is a unique identifying value in the security association. This value lets the receiving computer determine the security association to use to process the packet.
Packets Not Decrypted
The number of packets that the receiving IPsec driver cannot decrypt. A positive number may indicate one or more of the following problems:
- The security assosciation has expired.
- The security association is no longer valid.
- Authentication did not succeed.
- Integrity checking did not succeed.
Packets Not Authenticated
The number of packets that were not authenticated to the IPsec driver. A positive number may indicate that the security association has expired or is no longer valid. The IPsec driver must have the information in the security association in order to process the packets.
A positive number may also indicate that the two computers have incompatible authentication settings. Verify that the authentication method is the same for each computer.
The number of keys that the ISAKMP/Oakley mechanism sent to the IPsec driver. A positive number indicates that the ISAKMP phase two security associations were successfully negotiated.
ISAKMP/Oakley statistics are located in the lower-right window pane. The lower-right pane displays the following statistics for the ISAKMP/Oakley security mechanism:
Oakley main modes
The number of successful security associations that were established during ISAKMP phase one. A positive number indicates that the key information exchange was successful. Identities were authenticated and common keying material was established.
Oakley quick mode
The number of successful security associations that were established during ISAKMP phase two. A positive number indicates that the negotiation for protection services during the data transfer was successful.
The number of ISAKMP phase two negotiations that resulted in the computers agreeing only to a clear-text data transfer. A clear-text data transfer involves no encryption or signing of the packets.
The number of times that authentication of the computer identities did not succeed. If this number is positive, verify that the authentication method settings for each computer are compatible. A positive number may also indicate that the security association has expired.
A configurable option that lets you adjust the update rate of the data.
IP Security Monitor also indicates whether IP Security is enabled. This information is in the lower-right group box of the IP Security Monitor dialog box. To reset the statistics in IP Security Monitor, restart the IP Security Policy Agent by using Computer Management (Compmgmt.msc).
Using Network Monitor
You can use Network Monitor to analyze the following: • Network traffic
- The IKE exchange protocol
- The IPsec protocol
- The ESP protocol
- Authentication Header (AH)
Obtaining an Oakley log
Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:
322756 How to back up and restore the registry in Windows
Developers and network administrators who have advanced IKE knowledge can modify the registry to obtain an Oakley log. To do this, use Registry Editor to locate the following registry subkey. If the subkey does not exist, create it.
Add an entry of the REG_DWORD type value named "EnableLogging." Give this entry a value of 1. When this entry takes effect, an Oakley.log file is created in the %systemroot%\Debug folder.
Note To turn off logging, give the EnableLogging entry a value of 0.
Using the Netsh command
You can use the Netsh command to troubleshoot instances where IP offloading occurs on IPsec packets. IP offloading occurs when the network card instead of the CPU performs IP functions. For example, IP offloading occurs when the network card performs checksum calculations or performs packet encryption and decryption. IP offloading causes the IPsec driver to drop the packet. To determine whether an interface can perform IP offloading, follow these steps:
Click Start, click Run, type cmd, and then click OK.
At the command prompt, type netsh int ip show offload , and then press ENTER.
This command displays the offloading capabilities of the interface. However, the command does not display statistics. To view statistics, use IP Security Monitor to monitor confidential bytes received. Use these statistics to determine whether packets are lost or received.
To disable IP offloading, follow these steps to modify the registry:
- Click Start, click Run, type regedit , and then click OK.
- Locate and then click the following registry subkey:
If the EnableOffload entry is not present, follow these steps to create the entry:
- Right-click IPSEC, point to New, and then click DWORD value.
- Type EnableOffload to name the new value, and then press ENTER.
- Double-click EnableOffload.
- Type 0 , and then press ENTER.
If the IP connection that you are troubleshooting succeeds, the problem is caused by IP offloading.
Troubleshooting "bad SPI" messages in the Event Viewer
Restarting the policy agent
When you restart the policy agent, you remove old or nonsecure security associations. Restart the policy agent if IP Security Monitor does not show any security negotiations. Also restart the policy agent if you want to download a policy from the domain or from the policy store.
Verifying policy integrity
Active Directory assumes that the most recent changes are current. However, if multiple administrators try to change a policy at the same time, the links between policy components may break. A policy integrity check resolves this problem by verifying the links in all IPsec policies. Run an integrity check after any modifications are made to a policy. To test IPsec policy integrity, follow these steps:
- Click Start, click Run, type secpol.msc and then click OK.
- Right-click IP Security Policies on the Local Machine, point to All Tasks, and then click Check Policy Integrity.
Reviewing the IPsec driver and policy agent registry settings
Article ID: 257225 - Last Review: Jun 16, 2017 - Revision: 7