Article ID: 2664375 - View products that this article applies to.
Expand all | Collapse all

On This Page

Symptoms

Assume that you configure Routing and Remote Access Services (RRAS) virtual private network (VPN) server on a computer that is running Windows Server 2008 R2. However, when you try to connect to the VPN by using the L2TP/IPsec protocol, you cannot connect to the VPN successfully.

Your VPN client may report the following error message:

789
The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer.

Note This issue typically occurs when the Maximum Transmission Unit (MTU) value for the connection is smaller than the payload of the digital certificate for the IPsec connection. By default, the payload of the certificate is between 4,000 and 5,000 bytes.

The same problem might happen if you enable the VPN Client Access feature and allow L2TP/IPsec protocol for VPN clients in Microsoft Threat Management Gateway 2010 Server on a computer that is running Windows Server 2008 R2. Microsoft Threat Management Gateway 2010 (TMG) relies on the Routing and Remote Access Services (RRAS) component of the operating system.

Cause

This issue occurs because the VPN server rejects Internet Key Exchange (IKE) fragments from the VPN client after it moves into the IKE complete state.

Resolution

Hotfix information

A supported hotfix is available from Microsoft. However, this hotfix is intended to correct only the problem that is described in this article. Apply this hotfix only to systems that are experiencing the problem described in this article. This hotfix might receive additional testing. Therefore, if you are not severely affected by this problem, we recommend that you wait for the next software update that contains this hotfix.

If the hotfix is available for download, there is a "Hotfix download available" section at the top of this Knowledge Base article. If this section does not appear, contact Microsoft Customer Service and Support to obtain the hotfix.

Note If additional issues occur or if any troubleshooting is required, you might have to create a separate service request. The usual support costs will apply to additional support questions and issues that do not qualify for this specific hotfix. For a complete list of Microsoft Customer Service and Support telephone numbers or to create a separate service request, visit the following Microsoft website:
http://support.microsoft.com/contactus/?ws=support
Note The "Hotfix download available" form displays the languages for which the hotfix is available. If you do not see your language, it is because a hotfix is not available for that language.

Prerequisites

To apply this hotfix, you must be running Windows Server 2008 R2 Service Pack 1 (SP1). For more information about how to obtain a Windows Server 2008 R2 service pack, click the following article number to view the article in the Microsoft Knowledge Base:
976932 Information about Service Pack 1 for Windows 7 and for Windows Server 2008 R2

Registry information

To use the hotfix, you do not have to change the registry.

Restart requirement

You must restart the computer after you apply this hotfix.

Hotfix replacement information

This hotfix does not replace a previously released hotfix.

File information

The global version of this hotfix installs files that have the attributes that are listed in the following tables. The dates and the times for these files are listed in Coordinated Universal Time (UTC). The dates and the times for these files on your local computer are displayed in your local time together with your current daylight saving time (DST) bias. Additionally, the dates and the times may change when you perform certain operations on the files.
Windows Server 2008 R2 file information notes
Important Windows 7 hotfixes and Windows Server 2008 R2 hotfixes are included in the same packages. However, hotfixes on the Hotfix Request page are listed under both operating systems. To request the hotfix package that applies to one or both operating systems, select the hotfix that is listed under "Windows 7/Windows Server 2008 R2" on the page. Always refer to the "Applies To" section in articles to determine the actual operating system that each hotfix applies to.
  • The files that apply to a specific product, milestone (RTM, SPn), and service branch (LDR, GDR) can be identified by examining the file version numbers as shown in the following table:
    Collapse this tableExpand this table
    VersionProductMilestoneService branch
    6.1.760 1.21xxxWindows Server 2008 R2SP1LDR
  • The MANIFEST files (.manifest) that are installed for each environment are listed separately in the "Additional file information for Windows Server 2008 R2" section. MANIFEST files and the associated security catalog (.cat) files, are extremely important to maintain the state of the updated components. The security catalog files, for which the attributes are not listed, are signed with a Microsoft digital signature.
For all supported x64-based versions of Windows Server 2008 R2
Collapse this tableExpand this table
File nameFile versionFile sizeDateTimePlatform
Bfe.dll6.1.7601.21912706,56001-Feb-201206:29x64
Fwpuclnt.dll6.1.7601.21912324,09601-Feb-201206:30x64
Ikeext.dll6.1.7601.21912854,52801-Feb-201206:31x64
Networksecurity-ppdlic.xrm-msNot applicable3,02801-Feb-201206:54Not applicable
Nshwfp.dll6.1.7601.21912832,00001-Feb-201206:34x64
Wfp.mofNot applicable82210-Jun-200920:51Not applicable
Fwpuclnt.dll6.1.7601.21912216,57601-Feb-201206:47x86
Nshwfp.dll6.1.7601.21912657,92001-Feb-201206:50x86
Wfp.mofNot applicable82212-Nov-201023:57Not applicable
For all supported IA-64-based versions of Windows Server 2008 R2
Collapse this tableExpand this table
File nameFile versionFile sizeDateTimePlatform
Bfe.dll6.1.7601.219121,074,17601-Feb-201205:18IA-64
Fwpuclnt.dll6.1.7601.21912566,27201-Feb-201205:19IA-64
Ikeext.dll6.1.7601.219121,486,84801-Feb-201205:20IA-64
Networksecurity-ppdlic.xrm-msNot applicable3,02801-Feb-201205:48Not applicable
Nshwfp.dll6.1.7601.219121,114,11201-Feb-201205:22IA-64
Wfp.mofNot applicable82210-Jun-200920:57Not applicable
Fwpuclnt.dll6.1.7601.21912216,57601-Feb-201206:47x86
Nshwfp.dll6.1.7601.21912657,92001-Feb-201206:50x86
Wfp.mofNot applicable82212-Nov-201023:57Not applicable

Workaround

To work around this issue, connect to the VPN by using the PPTP or SSTP protocol.

Status

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

More information

For more information about software update terminology, click the following article number to view the article in the Microsoft Knowledge Base:
824684 Description of the standard terminology that is used to describe Microsoft software updates

Additional file information

Additional file information for Windows Server 2008 R2

Additional files for all supported x64-based versions of Windows Server 2008 R2
Collapse this tableExpand this table
File nameAmd64_861a8b129a6b7529965f08860ffc259d_31bf3856ad364e35_6.1.7601.21912_none_dac05b1d9600a1c6.manifest
File versionNot applicable
File size708
Date (UTC)01-Feb-2012
Time (UTC)09:26
PlatformNot applicable
File nameAmd64_c2ad5cfd38518b85e360928efb38bfc6_31bf3856ad364e35_6.1.7601.21912_none_a68de9ce422956e9.manifest
File versionNot applicable
File size1,058
Date (UTC)01-Feb-2012
Time (UTC)09:26
PlatformNot applicable
File nameAmd64_microsoft-windows-network-security_31bf3856ad364e35_6.1.7601.21912_none_2bd217415a49c6db.manifest
File versionNot applicable
File size154,831
Date (UTC)01-Feb-2012
Time (UTC)07:56
PlatformNot applicable
File nameWow64_microsoft-windows-network-security_31bf3856ad364e35_6.1.7601.21912_none_3626c1938eaa88d6.manifest
File versionNot applicable
File size94,348
Date (UTC)01-Feb-2012
Time (UTC)07:10
PlatformNot applicable
Additional files for all supported IA-64-based versions of Windows Server 2008 R2
Collapse this tableExpand this table
File nameIa64_017e7793b89d2aac75e93d68b1f688ac_31bf3856ad364e35_6.1.7601.21912_none_9b98772ebd2e5f7a.manifest
File versionNot applicable
File size1,056
Date (UTC)01-Feb-2012
Time (UTC)09:26
PlatformNot applicable
File nameIa64_microsoft-windows-network-security_31bf3856ad364e35_6.1.7601.21912_none_cfb51fb3a1ea5ea1.manifest
File versionNot applicable
File size154,828
Date (UTC)01-Feb-2012
Time (UTC)08:01
PlatformNot applicable
File nameWow64_microsoft-windows-network-security_31bf3856ad364e35_6.1.7601.21912_none_3626c1938eaa88d6.manifest
File versionNot applicable
File size94,348
Date (UTC)01-Feb-2012
Time (UTC)07:10
PlatformNot applicable

Properties

Article ID: 2664375 - Last Review: December 4, 2012 - Revision: 3.0
Applies to
  • Windows Server 2008 R2 Datacenter
  • Windows Server 2008 R2 Enterprise
  • Windows Server 2008 R2 for Itanium-Based Systems
  • Windows Server 2008 R2 Foundation
  • Windows Server 2008 R2 Standard
  • Windows Web Server 2008 R2
Keywords: 
kbautohotfix kbqfe kbhotfixserver kbfix kbexpertiseadvanced kbsurveynew KB2664375

Give Feedback

 

Contact us for more help

Contact us for more help
Connect with Answer Desk for expert help.
Get more support from smallbusiness.support.microsoft.com