IX509Enrollment::Enroll may encounter an access violation under certain conditions


Symptoms


IX509Enrollment::Enroll may encounter an access violation if the application cannot bind to the Key Isolation Storage service. The following stack causes the exception:

ncrypt!UnloadProvider
ncrypt!NCryptOpenStorageProvider+0xdd
certenroll!myNCOpenStorageProvider+0x41
certenroll!CertEnroll::CCspInformation::p_NCInitialize+0x4c
certenroll!CertEnroll::CCspInformation::p_Initialize+0x14
certenroll!CertEnroll::CCspInformation::InitializeFromName+0x7f
certenroll!CertEnroll::CCspInformations::p_AddCNGCsps+0x12f
certenroll!CertEnroll::CCspInformations::AddAvailableCsps+0x48
certenroll!CertEnroll::CX509EnrollmentManager::p_ExamX509Enrollments+0x3e8
certenroll!CertEnroll::CX509EnrollmentManager::p_ProcessX509Enrollments+0xa9
certenroll!CertEnroll::CX509EnrollmentManager::_EnrollThreadProc+0x3eb
certenroll!CertEnroll::CX509EnrollmentManager::_Enroll+0x115
certenroll!CertEnroll::CX509Enrollment::_EnrollWizard+0xca
certenroll!CertEnroll::CX509Enrollment::Enroll

Cause


By default, IX509Enrollment::Enroll will enumerate all legacy CSPs and CNG Providers. For CNG Providers, NCryptOpenStorageProvider is called to determine the algorithms that are supported by that KSP. The "Microsoft Smart Card Key Storage Provider" attempts to bind to the Isolated Storage service. If the binding fails, scksp!KspOpenStorageProvider fails and NCryptOpenStorageProvider will unload the scksp DLL which has a worker thread waiting on smart card events. When the DLL is unloaded the worker thread has no code to return to which causes an access violation.

Here is the worker thread before ncrypt!UnloadProvider is called:


ntdll!ZwWaitForMultipleObjects+0xa
kernel32!WaitForMultipleObjectsEx+0x10b
kernel32!WaitForMultipleObjects+0x11
scksp!I_TransactionManagerThreadProc+0xba
ntdll!TppWorkpExecuteCallback+0x1aa
ntdll!TppWorkerThread+0x3d6
kernel32!BaseThreadInitThunk+0xd
ntdll!RtlUserThreadStart+0x1d



Here is the worker thread after ncrypt!UnloadProvider unloads the scksp DLL:


kernel32!WaitForMultipleObjectsEx+0x111
kernel32!WaitForMultipleObjects+0x11
0x7fe`e4a27fb2

The thread returns to non-existing code.

Resolution


There are some ways to mitigate this problem.
1. Restarting the  “CNG Key Isolation” should reset the service which was in a bad state. This eliminates the Key Storage Provider failure.

2. The enumeration of the "Microsoft Smart Card Key Storage Provider" can programmatically be avoided by removing the smart card key storage provider from the CCspInformations property of the IX509CertificateRequest instance. Below is a code snippet that demonstrates this:
[DllImport("ncrypt.dll", SetLastError = true)] 
public static extern uint NCryptEnumStorageProviders(
out uint pImplCount,
out IntPtr ppImplList,
uint dwFlags);
[DllImport("ncrypt.dll", SetLastError = true)]
public static extern uint NCryptFreeBuffer(
IntPtr pvInput);
[DllImport("advapi32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
public static extern bool CryptEnumProviders(
uint dwIndex,
IntPtr pdwReserved,
uint dwFlags,
out uint pdwProvType,
IntPtr pszProvName,
ref uint pcbProvName);
[StructLayout(LayoutKind.Sequential)]
public class NCryptProviderName {
[MarshalAs(UnmanagedType.LPWStr)] public string pszName;
[MarshalAs(UnmanagedType.LPWStr)] public string pszComment;
};

CX509CertificateRequestPkcs10 objPkcs10 = newCX509CertificateRequestPkcs10Class();
CCspInformations objCSPs = new CCspInformationsClass();
bool fLoop = true;
uint index = 0;
IntPtr ptrCspName = Marshal.AllocHGlobal(512);
uint status;
uint count;
NCryptProviderName ProviderList = new NCryptProviderName();
IntPtr ptrProviderList;
do
{
uint dwProvName = 512;
uint dwProvType;
fLoop = CryptEnumProviders(index++, IntPtr.Zero, 0, out dwProvType, ptrCspName, ref dwProvName);
if (fLoop)
{
CCspInformation objCSP = new CCspInformationClass();
objCSP.InitializeFromName(Marshal.PtrToStringUni(ptrCspName));
objCSPs.Add(objCSP);
}
} while (fLoop);
Marshal.FreeHGlobal(ptrCspName);
status = NCryptEnumStorageProviders(out count, out ptrProviderList, 0);
if (status == 0)
{
for (int n = 0; n < count; n++)
{
Marshal.PtrToStructure(ptrProviderList, ProviderList);

if (ProviderList.pszName != "Microsoft Smart Card Key Storage Provider")
{
CCspInformation objCSP = new CCspInformationClass();
objCSP.InitializeFromName(ProviderList.pszName);
objCSPs.Add(objCSP);
}
ptrProviderList = (IntPtr)((IntPtr.Size == 4 ? ptrProviderList.ToInt32() : ptrProviderList.ToInt64()) + Marshal.SizeOf(ProviderList));
}
NCryptFreeBuffer(ptrProviderList);
}
objPkcs10.CspInformations = objCSPs;

objPkcs10.InitializeFromTemplateName(X509CertificateEnrollmentContext.ContextMachine, "MyTemplate");