Symptoms
After you install and upgrade to Microsoft Exchange Server 2016 Cumulative Update 6 (CU6), you can't access Outlook Web App (OWA) or Exchange Control Panel (ECP), and you receive the following error message:
:-( Something went wrong
We can't get that information right now. Please try again later.
X-ClientId: ClientID
X-FEServer: Exch1
In addition, the following events information is recorded in the Application log of the Exchange server that hosts the mailbox database:
Log Name: Application
Source: MSExchange OAuth
Event ID: 2004
Task Category: Configuration
Level: Warning
Keywords: Classic
User: N/A
Computer: mail.contoso.com
Description:
Unable to find the certificate with thumbprint CertificateValue in the current computer or the certificate is missing private key. The certificate is needed to sign the outgoing token.
Log Name: Application
Source: ASP.NET 4.0.30319.0
Event ID: 1309
Task Category: Web Event
Level: Warning
Keywords: Classic
User: N/A
Description:
Event code: 3005
Event message: An unhandled exception has occurred.
Event ID: EventID
Event sequence: 2
Event occurrence: 1
Event detail code: 0
Application information:
Application domain: /LM/W3SVC/2/ROOT/owa-2-DomainID
Trust level: Full
Application Virtual Path: /owa
Application Path: C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\owa\
Machine name: Exch1
Process information:
Process ID: 21508
Process name: w3wp.exe
Account name: NT AUTHORITY\SYSTEM
Exception information:
Exception type: TargetInvocationException
Exception message: Exception has been thrown by the target of an invocation.
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Owin.Loader.DefaultLoader.<>c__DisplayClass12.<MakeDelegate>b__b(IAppBuilder builder)
at Owin.Loader.DefaultLoader.<>c__DisplayClass1.<LoadImplementation>b__0(IAppBuilder builder)
at Microsoft.Owin.Host.SystemWeb.OwinAppContext.Initialize(Action`1 startup)
at Microsoft.Owin.Host.SystemWeb.OwinBuilder.Build(Action`1 startup)
at Microsoft.Owin.Host.SystemWeb.OwinHttpModule.InitializeBlueprint()
at System.Threading.LazyInitializer.EnsureInitializedCore[T](T& target, Boolean& initialized, Object& syncLock, Func`1 valueFactory)
at Microsoft.Owin.Host.SystemWeb.OwinHttpModule.Init(HttpApplication context)
at System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr appContext, HttpContext context, MethodInfo[] handlers)
at System.Web.HttpApplication.InitSpecial(HttpApplicationState state, MethodInfo[] handlers, IntPtr appContext, HttpContext context)
at System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr appContext, HttpContext context)
at System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext)
Encryption certificate is absent
at Microsoft.Exchange.Security.Authentication.Utility.GetCertificates()
at Microsoft.Exchange.Clients.Owa2.Server.Core.notifications.SignalR.SignalRStartup.Configuration(IAppBuilder app)
Request information:
Request URL: https://mail.contoso.com:URLID/owa/?bO=1
Request path: /owa/
User host address: UserHostAddressC:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\owa\
User:
Is authenticated: False
Authentication Type:
Thread account name: NT AUTHORITY\SYSTEM
Thread information:
Thread ID: 24
Thread account name: NT AUTHORITY\SYSTEM
Is impersonating: False
Stack trace: at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Owin.Loader.DefaultLoader.<>c__DisplayClass12.<MakeDelegate>b__b(IAppBuilder builder)
at Owin.Loader.DefaultLoader.<>c__DisplayClass1.<LoadImplementation>b__0(IAppBuilder builder)
at Microsoft.Owin.Host.SystemWeb.OwinAppContext.Initialize(Action`1 startup)
at Microsoft.Owin.Host.SystemWeb.OwinBuilder.Build(Action`1 startup)
at Microsoft.Owin.Host.SystemWeb.OwinHttpModule.InitializeBlueprint()
at System.Threading.LazyInitializer.EnsureInitializedCore[T](T& target, Boolean& initialized, Object& syncLock, Func`1 valueFactory)
at Microsoft.Owin.Host.SystemWeb.OwinHttpModule.Init(HttpApplication context)
at System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr appContext, HttpContext context, MethodInfo[] handlers)
at System.Web.HttpApplication.InitSpecial(HttpApplicationState state, MethodInfo[] handlers, IntPtr appContext, HttpContext context)
at System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr appContext, HttpContext context)
at System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext)
Cause
This issue occurs if the Exchange server Auth certificate that's used for OAuth signing is missing from the Exchange server. You can run the following command to check whether the certificate is missing:
Get-ExchangeCertificate (Get-AuthConfig).CurrentCertificateThumbprint
If the certificate is missing, you will receive the following error message:
A special Rpc error occurs on server Exch1: The certificate with thumbprint CertificateValue was not found.
+ CategoryInfo : NotSpecified: (:) [Get-ExchangeCertificate], InvalidOperationException
+ FullyQualifiedErrorId : [Server=Exch1,RequestId=RequestID] [FailureCategory=Cmdlet-InvalidOperationException] CEA009BC,Microsoft.Exchange.Management.SystemConfigurationTasks.GetExchangeCertificate
Resolution
To fix this issue, install Cumulative Update 7 for Exchange Server 2016 or a later cumulative update for Exchange Server 2016.
Workaround
If your organization has multiple Exchange servers, run the following command in the Exchange Management Shell to confirm if the OAuth certificate is present on other Exchange servers:
Get-ExchangeCertificate (Get-AuthConfig).CurrentCertificateThumbprint
If the certificate is present on other Exchange servers, export the certificate and then import it to the Exchange server that has the issue.
If the certificate isn't present on all Exchange servers in the organization, follow these steps to create and deploy a new OAuth certificate to the Exchange server:
-
Create a new OAuth certificate by running the following command:
New-ExchangeCertificate -KeySize 2048 -PrivateKeyExportable $true -SubjectName "cn=Microsoft Exchange Server Auth Certificate" -FriendlyName "Microsoft Exchange Server Auth Certificate" -DomainName "contoso.com"
Note Change the value of the DomainName parameter in the example (contoso.com) to the SMTP domain that's used in your organization.
-
Set the created certificate to be used for server authentication by running the following commands:
Set-AuthConfig -NewCertificateThumbprint <ThumbprintFromStep1> -NewCertificateEffectiveDate (Get-Date)
Set-AuthConfig –PublishCertificate
Set-AuthConfig -ClearPreviousCertificate -
Restart the Microsoft Exchange Service Host Service.
-
Either run the IISReset command to restart IIS or run the following commands (in elevated mode) to recycle OWA and ECP APP pools:
Restart-WebAppPool MSExchangeOWAAppPool
Restart-WebAppPool MSExchangeECPAppPoolNote In some environments, it may take an hour for the OAuth certificate to be published.
Status
Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.
References
Learn about the terminology that Microsoft uses to describe software updates.