"Error 789" error message when you use an L2TP VPN client through a Windows 2000 Terminal Services client session

Article translations Article translations
Article ID: 326751 - View products that this article applies to.
This article was previously published under Q326751
Notice
This article applies to Windows 2000. Support for Windows 2000 ends on July 13, 2010. The Windows 2000 End-of-Support Solution Center is a starting point for planning your migration strategy from Windows 2000. For more information see the Microsoft Support Lifecycle Policy.
Expand all | Collapse all

SYMPTOMS

When you try to use a virtual private network (VPN) connection through a Microsoft Windows 2000 Terminal Services client session, you may receive the following error message:
Error: 789 "The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer"
However, you can successfully establish the VPN connection from the Terminal server console.

CAUSE

This issue may occur if both of the following conditions are true:
  • You establish a client session by using Windows 2000 Terminal Services.
  • You try to establish a VPN connection by using Layer Two Tunneling Protocol (L2TP) from the Terminal server to connect to a Windows 2000 Server-based computer that is configured as a VPN server.

RESOLUTION

To resolve this issue, you can use a preshared key on both ends of the VPN connection. The L2TP/IPSec feature supports gateway-to-gateway VPN implementations by using a preshared key for Internet Key Exchange (IKE) authentication.

Note Microsoft does not support or recommend the use of a preshared key for IKE authentication on remote access L2TP/IPSec client connections. However, Windows 2000 is compliant with IKE Request for Comments (RFC) 2409.

MORE INFORMATION

To implement the preshared key authentication method for use with an L2TP/IPSec connection, follow these steps:
  1. Add the ProhibitIpSec registry value to both Windows 2000 Server-based endpoint computers, and then restart both computers.
  2. Manually configure an IPSec policy on both the Windows 2000 Server-based computers before you try to establish an L2TP/IPSec connection between them through a Terminal Services client session.

STATUS

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section of this article.

REFERENCES

For additional information about configuring L2TP, click the following article numbers to view the articles in the Microsoft Knowledge Base:
240262 How to configure a L2TP/IPSec connection using pre-shared key authentication
248711 Mutual authentication methods supported for L2TP/IPSec

Properties

Article ID: 326751 - Last Review: October 30, 2006 - Revision: 1.3
APPLIES TO
  • Microsoft Windows 2000 Advanced Server
  • Microsoft Windows 2000 Server
Keywords: 
kbprb KB326751

Give Feedback

 

Contact us for more help

Contact us for more help
Connect with Answer Desk for expert help.
Get more support from smallbusiness.support.microsoft.com