SQL Server Connection Fails When RC4 Encryption Is Disabled

View products that this article applies to.

This article was previously published under Q324914

SYMPTOMS

You may disable various Schannel encryption algorithms such as RC4 to enforce FIPS 140-1 compliant encryption on a computer that is running Microsoft Windows. If you have a server that is running SQL Server on this computer, and if you enable protocol encryption, you cannot connect to the server that is running SQL Server from a client computer. When you try to connect, you receive the following error message on the client computer:

Communication link failure.

Back to the top

RESOLUTION

A supported fix is now available from Microsoft, but it is only intended to correct the problem described in this article. Only apply it to systems that are experiencing this specific problem. This fix may receive additional testing to further ensure product quality. Therefore, if you are not severely affected by this problem, Microsoft recommends that you wait for the next SQL Server 2000 and Microsoft Data Access Components (MDAC) service pack that contains this hotfix.

To resolve this problem immediately, contact Microsoft Product Support Services to obtain the fix. For a complete list of Microsoft Product Support Services phone numbers and information about support costs, visit the following Microsoft Web site:

http://support.microsoft.com/default.aspx?scid=fh;EN-US;CNTACTMS (http://support.microsoft.com/default.aspx?scid=fh;en-us;cntactms)

Note In special cases, charges that are ordinarily incurred for support calls may be canceled if a Microsoft Support Professional determines that a specific update will resolve your problem. The usual support costs will apply to additional support questions and issues that do not qualify for the specific update in question.

The English version of this fix has the file attributes (or later) that are listed in the following table. The dates and times for these files are listed in coordinated universal time (UTC). When you view the file information, it is converted to local time. To find the difference between UTC and local time, use the Time Zone tab in the Date and Time tool in Control Panel.

MDAC 2.7 and SQL Server 2000 Service Pack 2 (SP2)

   Date         Time   Version            Size    File name
   --------------------------------------------------------------
   03-Jul-2002  16:02  2000.81.9001.9     61,440  Dbnetlib.dll
   02-Jul-2002  17:54  2000.80.653.0      82,492  Ssnetlib.dll

MDAC 2.7 SP1 and SQL Server 2000 Service Pack 3 (SP3)

   Date         Time   Version            Size    File name
   --------------------------------------------------------------
   21-Feb-2003  16:02  2000.81.9031     61,440  Dbnetlib.dll

Back to the top

WORKAROUND

To work around this problem, either enable RC4 encryption, or disable the Force Protocol Encryption option for the server that is running SQL Server. However, to use FIPS 140-1 compliant encryption with SQL Server 2000, you must install the hotfix that is listed in the "Resolution" section.

Back to the top

STATUS

Microsoft has confirmed that this is a problem in the Microsoft products that are listed at the beginning of this article.

Back to the top

MORE INFORMATION

Steps to Reproduce the Behavior

To reproduce this problem on a computer that is running Microsoft Windows 2000, follow these steps:

Make sure that SQL Server 2000 SP2 is installed.
Follow the steps that are detailed in the following Microsoft Knowledge Base article in the "FIPS 140-1 Cipher Suites" section to force Windows 2000 to use only FIPS 140-1 cipher suites:
245030 (http://support.microsoft.com/kb/245030/EN-US/ ) How to Restrict the Use of Certain Cryptographic Algorithms and Protocols in Schannel.dll
Restart the computer to make sure that the registry changes and the encryption options are fully enabled on the computer.
Configure a certificate by using the typical certificate setup procedure for SQL Server to allow protocol encryption.
In the Server Network Utility, make sure that the Force Protocol Encryption check box is selected for the appropriate instance of SQL Server to enforce encryption for that instance.
Stop and restart SQL Server 2000, and then verify in the SQL Server error log that encryption is enabled.
From a client computer, run Query Analyzer, and then try to connect to the server that is running SQL Server. You receive the following error message in Query Analyzer:
Unable to connect to server SQL Server Name:
ODBC: Msg 0, Level 16, State 1
[Microsoft][ODBC SQL Server Driver]Communication link failure