Symptoms
Assume that you're using Transport Layer Security (TLS) protocol version 1.2 in Microsoft SQL Server. When the certificate that's used to encrypt the endpoint communication for database mirroring, availability groups, and service broker uses an MD5 hashing algorithm, communication fails. Additionally, you receive the following error message in SQL Server Error log:
Connection handshake failed. An OS call failed: (80090331) 0x80090331(The client and server cannot communicate, because they do not possess a common algorithm.). State 58.'.
Additionally, the Windows log reports the following Schannel error:
A fatal alert was generated and sent to the remote endpoint. This may result in termination of the connection. The TLS protocol defined fatal error code is 40. The Windows SChannel error state is 1205.
Cause
This issue occurs because TLS 1.2 doesn't support MD5 as a signature hash algorithm.
Resolution
Based on current security best practices and standards, we recommend that you switch to a non-MD5 signature hash for certificates that are used for SQL Server endpoint encryption.
The list of fixed versions that support TLS 1.2 is available in the following Microsoft Knowledge Base article:3135244 TLS 1.2 support for Microsoft SQL Server
Status
Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.
References
Learn about the terminology that Microsoft uses to describe software updates.