United States

Change

All Microsoft Sites

Microsoft Support

Article ID: 257225 - Last Review: October 12, 2007 - Revision: 7.3

IPsec troubleshooting in Microsoft Windows 2000 Server

View products that this article applies to.

This article was previously published under Q257225

SUMMARY

To troubleshoot IPsec connection problems in Microsoft Windows 2000, first verify the success of Internet Key Exchange (IKE) security negotiation. To do this, enable Audit policy and then examine the Security log. Next, use the Netdiag.exe command-line tool to display debugging information. Then, depending on whether the problem occurs in phase one or in phase two, examine your IPsec policy properties and IPsec rules.

Use IP Security Monitor to view more information about IPsec and security associations. You can also use IP Security Monitor to view IKE statistics. Use Network Monitor to analyze network traffic and the status of the various protocols used in your network. You can use the Netsh command to troubleshoot instances where IP offloading occurs on IPsec packets.

You can also use the information in this article to do the following:

Obtain an Oakley log
Understand the contents of event logs
Troubleshoot "bad SPI" messages
Restart the policy agent
Verify the integrity of your policies

Back to the top

INTRODUCTION

This article contains guidelines for troubleshooting Internet Protocol security (IPsec) connection problems in Microsoft Windows 2000. IPsec relies on the Internet Key Exchange (IKE) protocol to establish shared security parameters and authenticated keys between two computers. The IKE protocol uses two phases. In phase one, Windows 2000 uses the IKE Security Association and Key Management Protocol (ISAKMP) Main Mode exchange. (Windows 2000 does not support Aggressive Mode.) When the phase one exchange provides a secured channel, the computers obtain an authenticated key and an IKE security association. This secured channel is used in phase two to help protect the Quick Mode Exchange. The Quick Mode Exchange provides IPsec security associations.

Back to the top

MORE INFORMATION

Back to the top

Basic IPsec troubleshooting

To troubleshoot IPsec, first enable Audit policy, and then verify the results of phase one and phase two exchanges. When you enable Audit policy, security events are logged in the Security log. By examining the Security log, you can determine whether IKE security association negotiation is successful. To enable Audit policy, follow these steps:

In Group Policy, expand Local Computer Policy.
Locate and then click Computer Configuration/Windows Settings/Security Settings/Local Policies/Audit Policy.
In the details pane, right-click Audit logon events, and then click Security.
Click to select Success, click to select Failure, and then click OK.
In the details pane, right-click Audit object access, and then click Security.
Click to select Success, click to select Failure, and then click OK.

Note If you are using a domain policy for auditing, the domain policy overwrites your local policy.

Next, type the following command to use the Netdiag.exe command-line tool:

netdiag /test:ipsec /debug

This command displays debugging information about phase two.

Note To use Netdiag.exe, the Windows 2000 Support Tools package must be installed on your computer. To install the Windows 2000 Support Tools, follow these steps:

Start Windows 2000.

Note You must log on as a member of the administrator group to install these tools.
Insert the Windows 2000 CD into your CD drive.
Click Browse this CD, and then open the Support\Tools folder.
Double-click Setup.exe, and then follow the instructions that appear on the screen.

You can also use Netdiag.exe to view the policy without an active connection. To do this, type the following command at a command prompt, and then press ENTER:

netdiag /test:ipsec /v

This command displays the current policy and IPsec statistics with regard to phase one.

If the logged events indicate that phase one Main Mode exchange fails, verify the IKE settings and the IKE authentication methods in your IPsec policy properties. To do this, follow these steps:

Click Start, click Run, type secpol.msc, and then click OK.
Click the IPsec rule that you want to click, right-click IPsec rules and then click Properties.
Click the General tab, and then verify that the settings are correct.
Click Advanced, examine the settings, click Methods, and then examine the settings.
Click OK two times.
Click Rules tab, click Edit, and then click the Authentication Methods tab.
Examine the settings on this tab.

If the logged events indicate that phase two Quick Mode fails, verify the IPsec security methods in the IPsec rules and in your IPsec policy properties. To do this, follow these steps:

Click the IPsec rule that you want to verify, click Edit, and then click the Filter Action tab.
Click the filter action that is enabled, click Edit, and examine the settings.

Back to the top

Using IP Security Monitor

You can use IP Security Monitor to monitor your security associations, IPsec statistics, and IKE statistics. In particular, you can use IP Security Monitor to verify the success of authentication and security associations. To start IP Security Monitor, click Start, click Run, type ipsecmon, and then click OK.

Note By default, IP Security Monitor displays statistics for the local computer. To specify a remote computer, click Start, click Run, type ipsecmon computer_name, and then click OK.

The upper group box in the IP Security Monitor dialog box displays the active security associations and the configuration of the active policy. The lower left group box displays the following IPsec statistics:

Active Associations
The number of active security associations.
Confidential Bytes Sent
The number of bytes sent by using the Encapsulating Security Payload (ESP) security protocol (decimal ID 50).
Confidential Bytes Received
The number of bytes received by using the ESP security protocol.
Authenticated Bytes Sent
The number of bytes sent with the authentication property enabled.
Authenticated Bytes Received
The number of bytes received with the authentication property enabled.
Bad SPI Packets
The number of packets whose Security Parameters Index (SPI) is not valid. A positive number probably indicates that the security association has expired or is no longer valid.

The SPI is a unique identifying value in the security association. This value lets the receiving computer determine the security association to use to process the packet.
Packets Not Decrypted
The number of packets that the receiving IPsec driver cannot decrypt. A positive number may indicate one or more of the following problems:
- The security association has expired
- The security association is no longer valid
- Authentication did not succeed
- Integrity checking did not succeed
Packets Not Authenticated
The number of packets that were not authenticated to the IPsec driver. A positive number may indicate that the security association has expired or is no longer valid. The IPsec driver must have the information in the security association in order to process the packets.

A positive number may also indicate that the two computers have incompatible authentication settings. Verify that the authentication method is the same for each computer.
Key Additions
The number of keys that the ISAKMP/Oakley mechanism sent to the IPsec driver. A positive number indicates that the ISAKMP phase two security associations were successfully negotiated.

ISAKMP/Oakley statistics are located in the lower-right window pane. The lower-right pane displays the following statistics for the ISAKMP/Oakley security mechanism:

Oakley main modes
The number of successful security associations that were established during ISAKMP phase one. A positive number indicates that the key information exchange was successful. Identities were authenticated and common keying material was established.
Oakley quick mode
The number of successful security associations that were established during ISAKMP phase two. A positive number indicates that the negotiation for protection services during the data transfer was successful.
Soft Associations
The number of ISAKMP phase two negotiations that resulted in the computers agreeing only to a clear-text data transfer. A clear-text data transfer involves no encryption or signing of the packets.
Authentication failures
The number of times that authentication of the computer identities did not succeed. If this number is positive, verify that the authentication method settings for each computer are compatible. A positive number may also indicate that the security association has expired.
IPsecmon configurations
A configurable option that lets you adjust the update rate of the data.

IP Security Monitor also indicates whether IP Security is enabled. This information is in the lower-right group box of the IP Security Monitor dialog box. To reset the statistics in IP Security Monitor, restart the IP Security Policy Agent by using Computer Management (Compmgmt.msc).

Back to the top

Using Network Monitor

You can use Network Monitor to analyze the following:

Network traffic
The IKE exchange protocol
The IPsec protocol
The ESP protocol
Authentication Header (AH)

Back to the top

Obtaining an Oakley log

Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:

322756 (http://support.microsoft.com/kb/322756/ ) How to back up and restore the registry in Windows

Developers and network administrators who have advanced IKE knowledge can modify the registry to obtain an Oakley log. To do this, use Registry Editor to locate the following registry subkey. If the subkey does not exist, create it.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PolicyAgent\Oakley

Add an entry of the REG_DWORD type value named "EnableLogging." Give this entry a value of 1. When this entry takes effect, an Oakley.log file is created in the %systemroot%\Debug folder.

Note To turn off logging, give the EnableLogging entry a value of 0.

Back to the top

Using the Netsh command

You can use the Netsh command to troubleshoot instances where IP offloading occurs on IPsec packets. IP offloading occurs when the network card instead of the CPU performs IP functions. For example, IP offloading occurs when the network card performs checksum calculations or performs packet encryption and decryption. IP offloading causes the IPsec driver to drop the packet. To determine whether an interface can perform IP offloading, follow these steps:

Click Start, click Run, type cmd, and then click OK.
At the command prompt, type netsh int ip show offload, and then press ENTER.

This command displays the offloading capabilities of the interface. However, the command does not display statistics. To view statistics, use IP Security Monitor to monitor confidential bytes received. Use these statistics to determine whether packets are lost or received.

To disable IP offloading, follow these steps to modify the registry:

Click Start, click Run, type regedit, and then click OK.
Locate and then click the following registry subkey:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\IPSEC
If the EnableOffload entry is not present, follow these steps to create the entry:
1. Right-click IPSEC, point to New, and then click DWORD value.
2. Type EnableOffload to name the new value, and then press ENTER.
3. Double-click EnableOffload.
4. Type 0, and then press ENTER.

If the IP connection that you are troubleshooting succeeds, the problem is caused by IP offloading.

Back to the top

Event logs

The following events may be logged in the Security event log:

```
Informational event 279
Source: PolicyAgent 
Category: None
```
This event indicates the following:
- Whether an IPsec policy is in effect
- The source of the IPsec policy
- The Active Directory polling interval, if the source of the policy is a domain
Additionally, if an IPsec policy was changed, the event text includes "Updating IPsec Policy."
```
Error event 284
Source: PolicyAgent 
Category: None
```
This event indicates that the IP Security Policy Agent cannot contact the Active Directory for the domain that the computer belongs to.
```
Audit event 541
Source: Security 
Category: Logon/Logoff
```
This event indicates that an IPsec hard security association has been established. Soft security associations are not audited.
```
Audit event 542
Source: Security 
Category: Logon/Logoff
```
This event indicates that an IPsec security association has ended. The ended security association may be hard or soft.

The following events may be logged in the Application event log. The Application event log includes messages from ISAKMP/Oakley. The following events indicate that a domestic version of Windows 2000 tried to negotiate greater security than an export client can support.

```
Warning event 541
Source: Oakley 
Category: None
```
This event indicates that the export client cannot generate domestic-strength key material. The resulting negotiation agrees only on export-strength key material.
```
Warning event 542
Source: Oakley 
Category: None
```
This event indicates that the export client cannot perform encryption stronger than Data Encryption Standard (DES). The resulting negotiation agrees only on DES if the other computer can support DES.

Back to the top

General troubleshooting

Troubleshooting "bad SPI" messages in the Event Viewer

"Bad SPI" messages are logged in the following circumstances:

If a key lifetime value is set too low
If the sender continues to transmit data to the receiver after the security association has expired

When you receive a new security association, you must start transmitting data on it. However, if you communicate with a slower responder, the slower responder may receive IPSed-protected data that it does not recognize. The responder considers an SPI that it does not recognize a "bad SPI." To determine the problem and to correct it, use IP Security Monitor to examine the number of rekeys. Consider how long the connections have been active. If the number of rekeys is very large, configure longer key lifetimes in the policy. Acceptable values for high-traffic Ethernet connections are more than 50 megabytes and more than five minutes.

Configuring longer values may not prevent bad SPIs. However, configuring longer values can significantly reduce the number of bad SPIs. Typically, Windows 2000 Server logs event 4268 to indicate that packets were discarded because of a bad SPI.

If IP Security Monitor indicates that secured security associations are not established, nonsecure security associations may be preventing secured security associations from being established.

Note A secured security association is also known as a hard security association. An nonsecure security association is also known as a soft security association.

Run IP Security Monitor on one of the peer computers. If a security association exists, and the security setting is None, an nonsecure security association exists. An nonsecure security association remains on the computer as long as traffic is regularly sent. To prevent this condition, stop all traffic until the security association times out. Typically, the security association times out in five minutes. Use IP Security Monitor to make sure that the security association is no longer established, and then start traffic again. If policies are compatible, a secured security association is automatically established. Restart the policy agent to delete all nonsecure security associations.

If the files that are required for IPsec components have been removed or deleted, reinstall the IPsec components by removing and then reinstalling the TCP/IP network protocol. Files that IPsec components require include the following:

ISAKMP/Oakley
IPsec Policy Agent
The IPsec driver

IPsec components are reinstalled when you reinstall TCP/IP.

IPsec negotiations may fail because of incompatible IPsec policy settings. Examine the Security event log on each computer that participates in a negotiation. Recent events may record attempts to perform an Oakley negotiation. The events may include a description of the success or the failure.

Verify the integrity of the policy on each computer. To determine the cause of a policy mismatch, follow these steps:

Make sure that the authentication methods are compatible.
Make sure that at least one compatible security method is correctly configured.
If you use IPsec tunneling, make sure that the tunnel endpoint settings are correct. Also make sure that the endpoint computers are functioning correctly.

Note The tunnel endpoint settings include settings for ISAKMP/Oakley, the IPsec Policy Agent, and the IPsec driver.

If "Wrong IPsec policy location" appears in the event log, examine the last lines in the policy agent log. The log may indicate the location of the policy that was used. If the policy location is not logged, follow these steps:

Click Start, click Run, type cmd, and then click OK.
Type the following command, and then press ENTER:
findstr %systemroot%\ipsecpa.log
Note This command is case-sensitive.

The log indicates whether Group Policy settings or local computer policy settings are used.

Restarting the policy agent

When you restart the policy agent, you remove old or nonsecure security associations. Restart the policy agent if IP Security Monitor does not show any security negotiations. Also restart the policy agent if you want to download a policy from the domain or from the policy store.

Verifying policy integrity

Active Directory assumes that the most recent changes are current. However, if multiple administrators try to change a policy at the same time, the links between policy components may break. A policy integrity check resolves this problem by verifying the links in all IPsec policies. Run an integrity check after any modifications are made to a policy. To test IPsec policy integrity, follow these steps:

Click Start, click Run, type secpol.msc and then click OK
Right-click IP Security Policies on the Local Machine, point to All Tasks, and then click Check Policy Integrity.

Reviewing the IPsec driver and policy agent registry settings

The settings for the IPsec driver are located in the following registry subkey:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\IPsec

You can modify the values of the following entries:

SAIdleTime

This REG_DWORD entry configures the Security Association Idle Timer. The default value is 300 seconds. You can specify a value of 300 to 3600 seconds.
CacheSize

This REG_DWORD entry configures the IP header-based cache size. The default value is 64 KB. You can specify a value of 64 to 1024 KB.
SAHashSize

This REG_DWORD entry configures the size of the SPI. It also configures the destination table for inbound security associations. The default value is 64 KB. You can specify a value of 64 to 1024 KB.

The settings for the policy agent are located in the following registry subkey:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PolicyAgent

You can modify the values of the following entries:

Debug

The data type is REG_DWORD. The default value is 0. A value of 1 turns on logging. When logging is turned on, the Ipsecpa.log file is created in the %system root%\Debug folder.
Log

The data type is REG_SZ. This entry specifies the name of the log file to open when the Debug entry is set to 1.

The global IPsec policy references are located in the following registry subkeys:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PolicyAgent\Policy\GPTIPSECPolicy
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\IPsec\GPTIPSECPolicy
HKEY_CURRENT_USER\SOFTWARE\Microsoft\GroupPolicyObjects\{GUID}Machine\Software\Policies\Microsoft\Windows\IPsec\GPTIPSECPolicy
HKEY_CURRENT_USER\SOFTWARE\Microsoft\GroupPolicyObjects\{GUID}Machine\System\CurrentControlSet\Services\PolicyAgent\Policy\GPTIPSECPolicy
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\IPsec\GPTIPSECPolicy

Back to the top

REFERENCES

231585 (http://support.microsoft.com/kb/231585/ ) Overview of secure IP communication with IPsec in Windows 2000

For more information about Layer 2 Tunneling Protocol (L2TP)/IPsec connections, click the following article number to view the article in the Microsoft Knowledge Base:

248750 (http://support.microsoft.com/kb/248750/ ) Description of the IPSec policy created for L2TP/IPSec

Back to the top