Failed TLS connection between unified communications peers generates an Schannel warning

Article translations Article translations
Article ID: 2464556 - View products that this article applies to.
Expand all | Collapse all

On This Page

Symptoms

Each of the following applications cannot create a Transport Layer Security (TLS) connection with that application's unified communications (UC) peer:
  • Microsoft Exchange Server
  • Microsoft Office Communications Server
  • Microsoft Communicator client
  • Microsoft Office Live Meeting 2007 client
  • Microsoft Lync Server
  • Microsoft Lync client
Additionally, you receive the following Schannel warning and description of the issue in the Windows Server event log:

Event Type: Warning
Event Source: Schannel
Event Category: None
Event ID: 36885
Date: date
Time: time
User:
Computer: COMPUTERNAME

Description: When asking for client authentication, this server sends a list of trusted certification authorities to the client. The client uses this list to choose a client certificate that is trusted by the server. Currently, this server trusts so many certification authorities that the list has grown too long. This list has thus been truncated. The administrator of this machine should review the certification authorities trusted for client authentication and remove those that do not really need to be trusted.

Cause

This issue occurs because the UC server does not pass the correct certification authority information back to the UC client during the negotiation of the TLS connection. Instead, the following scenario occurs:

  • The UC server passes its certificate trust list (CTL) of installed certification authority information to the UC client that requests the secure TLS connection.
  • The CTL is truncated as per the design limitations of the Windows Server Schannel component.
  • The UC client that requested the secure TLS connection does not receive certification authority information that matches the entries that are contained in its installed certification authority list.
  • The TLS connection attempt fails with the error that is described in the "Symptoms" section.

This issue occurs because of the design of the Schannel component of the Windows Server operating system that is hosting the UC applications.

Note For more information about the Windows Server operating system’s Schannel component and the limitations of its CTL, refer to the "Windows Server 2003," "Windows Server 2008 R2 and Windows Server 2008," and "Windows Server 2012" subsections of the "More Information" section.

Workaround

The following methods work around the issue that is described in the "Symptoms" section.

Method 1: Remove some trusted root certificates


Warning You should use caution when you remove Trusted Root Authority certificates. The removal of third-party Trusted Root Authority certificates could break secure client access to applications that are hosted on the Windows-based server.

If some trusted root certificates are not used in your environment, you should remove them from the server that is hosting the UC application. To do this, follow these steps:

Windows Server 2008 R2, Windows Server 2008, and Windows Server 2003
  1. Click Start, click Run, type mmc, and then click OK.
  2. On the File menu, click Add/Remove Snap-in, and then click Add.
  3. In the Add Standalone Snap-in dialog box, click Certificates, and then click Add.
  4. Click Computer account, click Next, and then click Finish.
  5. Click Close, and then click OK.
  6. Under Console Root in the Microsoft Management Console (MMC) snap-in, expand Certificates (Local Computer), expand Trusted Root Certification Authorities, and then click Certificates.
  7. Remove trusted root certificates that you do not have to have. To do this, right-click a certificate, click Delete, and then click Yes to confirm that you want to remove the certificate.
InformationTo learn more about how to automate the removal and installation of the third-party Trusted Root Authority certificates that are installed on Windows Server operating systems, click the following article number to view the article in the Microsoft Knowledge Base:

2801679 SSL/TLS communication problems after you install KB 931125

Information There are some root certificates that are required by Windows. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

293781 Trusted root certificates that are required by Windows Server 2008 R2, Windows 7, Windows Server 2008, Windows Vista, Windows Server 2003, Windows XP, and Windows 2000

Method 2: Configure Group Policy to ignore the list of trusted certification authorities on the computer that hosts the UC client

If the server that hosts the UC application is a member of a domain, you can create a policy that causes the server to ignore the list of trusted certification authorities on the computer that hosts the UC client. When you apply this policy, affected servers and clients trust only certificates that are in the Enterprise Root Certification Authorities store. Therefore, you do not have to change individual computers.

Windows Server 2008 R2 or Windows Server 2008

Use Policy to Distribute Certificates

Windows Server 2003

Add a trusted root certification authority to a Group Policy object


Note There are some root certificates that are required by Windows. You must add these certificates to the policy that you created. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

293781 Trusted root certificates that are required by Windows Server 2008 R2, Windows 7, Windows Server 2008, Windows Vista, Windows Server 2003, Windows XP, and Windows 2000

Method 3: Configure Schannel to no longer send the list of trusted root certification authorities during the TLS/SSL handshake process

You can follow these steps in Windows Server 2008 R2, Windows Server 2008, and Windows Server 2003.

Warning Serious problems might occur if you modify the registry incorrectly by using Registry Editor or by using another method. These problems might require you to reinstall the operating system. Microsoft cannot guarantee that these problems can be resolved. Modify the registry at your own risk.

On the server that is running the UC application on which you experience this problem, set the following registry entry to false:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL

Value name: SendTrustedIssuerList
Value type: REG_DWORD
Value data: 0 (False)


By default, this entry is not listed in the registry. By default, this value is 1 (True). This registry entry controls the flag that controls whether the server sends a list of trusted certification authorities to the client. When you set this registry entry to False, the server does not send a list of trusted certification authorities to the client. This behavior may affect how the client responds to a request for a certificate. For example, if Internet Explorer receives a request for client authentication, Internet Explorer displays only the client certificates that appear in the chain of one of the certification authorities that are in the list from the server. However, if the server does not send a list of trusted certification authorities, Internet Explorer displays all the client certificates that are installed on the client computer.

To set this registry entry, follow these steps:
  1. Click Start, click Run, type regedit, and then click OK.
  2. Locate and then click the following registry subkey:
    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL
  3. On the Edit menu, point to New, and then click DWORD Value.
  4. Type SendTrustedIssuerList, and then press Enter to name the registry entry.
  5. Right-click SendTrustedIssuerList, and then click Modify.
  6. In the Value data box, type 0 if that value is not already displayed, and then click OK.
  7. Exit Registry Editor.
Note There are some root certificates that are required by Windows. You must add these certificates to the policy that you created. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

293781 Trusted root certificates that are required by Windows Server 2008 R2, Windows 7, Windows Server 2008, Windows Vista, Windows Server 2003, Windows XP, and Windows 2000

More information

Although this is not noted in the "Symptoms" section, this issue can occur on all computers that are running Windows Server 2003 or Windows Server 2008 and are hosting Microsoft Exchange Unified Communications components such as the following roles:
  • Client Access Server
  • Mailbox
  • Unified Communications Server

These Windows Server operating systems can use the automatic root update mechanism to download certification authority updates from the Microsoft Windows Update website when a client requires a secure TLS connection. This automated certificate update process causes the server to accumulate the additional certification authority information that is needed to make sure of secure UC client TLS connection requests. The Windows Server Schannel component marshals the root certification authority information to the UC client that requires the secure TLS connection. The UC client will compare the content of the Schannel CTL with its own list of certification authority information. This process makes sure that matching root certificate information is present at both endpoints of the TCP connection for the continuance of the TLS handshake process. However, the Windows Server Schannel component can marshal only a limited amount of the Windows Server installed certification authority information in its CTL back to the UC client that requests the secure TLS connection. In some scenarios, this causes the UC client's secure TLS connection request to fail.

For more information about design differences for the Windows Server 2003 and Windows Server 2008 automatic root update configurations and about Schannel certification authority list limitations, see the following sections.

Windows Server 2003


In Windows Server 2003, the issuer list cannot be larger than 12,288 (or 0x3000) bytes. The installation of Windows Server 2003 SP1 or of the Windows Server 2003 SP2 hotfix KB933940 that is listed in the "More Information" section provides an enlarged issuer list capacity of 16,384 (or 0x4000) bytes.

The automatic root update mechanism is not enabled in Windows Server 2003. Windows Server 2003 supports the automatic root update mechanism. For more information about automatic root updates for Server 2003, go to the following Microsoft TechNet website:

Turn on automatic updating of trusted root authority certificates

Windows Server 2008 R2 and Windows Server 2008


In Windows Server 2008, the issuer list cannot be larger than 16,384 (or 0x4000) bytes.

By default, the automatic root update mechanism is enabled in Windows Server 2008 and Windows Server 2008 R2. For more information about how to manage the automatic root update mechanism, go to the following Microsoft TechNet website:

Certificate Support and Resulting Internet Communication in Windows Server 2008

Windows Server 2012

Windows Server 2012 does not support the Schannel CTL functionality that was present in earlier versions of the Windows Server operating systems. For more information about how Windows Server 2012 manages trusted certificates, read the "Management of trusted issuers for client authentication" section of the following Microsoft TechNet article:
What's New in TLS/SSL (Schannel SSP)

For more information about the issue that is described in the "Symptoms" section, click the following article numbers to view the articles in the Microsoft Knowledge Base:

933430 Clients cannot make connections if you require client certificates on a website or if you use IAS in Windows Server 2003

931125 Windows root certificate program members

Properties

Article ID: 2464556 - Last Review: October 21, 2013 - Revision: 6.0
Applies to
  • Microsoft Lync Server 2013
  • Microsoft Lync Server 2010 Enterprise Edition
  • Microsoft Lync Server 2010 Standard Edition
  • Microsoft Office Communications Server 2007 R2 Enterprise Edition
  • Microsoft Office Communications Server 2007 R2 Standard Edition
  • Microsoft Office Communications Server 2007 Standard Edition
  • Microsoft Office Communications Server 2007 Enterprise Edition
  • Microsoft Exchange Server 2010 Enterprise
  • Microsoft Exchange Server 2007 Enterprise Edition
  • Microsoft Exchange Server 2010 Standard
  • Microsoft Exchange Server 2007 Standard Edition
  • Microsoft Office Live Meeting 2007
Keywords: 
kbsurveynew vkbportal107 vkbportal230 vkbportal238 kbtshoot kbexpertiseinter KB2464556

Give Feedback

 

Contact us for more help

Contact us for more help
Connect with Answer Desk for expert help.
Get more support from smallbusiness.support.microsoft.com