This issue occurs when you use a certificate whose serial number is greater than 16 bytes. When this issue occurs, you receive the following error message:
Msg 15297, Level 16, State 56, Line 1
The certificate, asymmetric key, or private key data is invalid.
The issue was first fixed in the following cumulative update of SQL Server:
Each new cumulative update for SQL Server contains all the hotfixes and all the security fixes that were included with the previous cumulative update. We recommend that you download and install the latest cumulative updates for SQL Server:
Note After you install this update, you can create the certificate even though its serial number is greater than 16 bytes without receiving any error message. However, the serial number will be truncated to 16 bytes when it's saved into the cert_serial_number column in the sys.certificates catalog view. The truncation occurs only in catalog view. Therefore, the certificate still preserves the original length for its serial number.
Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.