You can't access OWA or ECP after you install Exchange Server 2016 CU6

Applies to: Exchange Server 2016 Standard EditionExchange Server 2016 Enterprise Edition

Symptoms


After you install and upgrade to Microsoft Exchange Server 2016 Cumulative Update 6 (CU6), you can't access Outlook Web App (OWA) or Exchange Control Panel (ECP), and you receive the following error message:

:-( Something went wrong
We can't get that information right now. Please try again later.
X-ClientId: ClientID
X-FEServer: Exch1

In addition, the following events information is recorded in the Application log of the Exchange server that hosts the mailbox database:

Log Name: Application
Source: MSExchange OAuth
Event ID: 2004
Task Category: Configuration
Level: Warning
Keywords: Classic
User: N/A
Computer: mail.contoso.com
Description:
Unable to find the certificate with thumbprint CertificateValue in the current computer or the certificate is missing private key. The certificate is needed to sign the outgoing token.
Log Name:      Application
Source:        ASP.NET 4.0.30319.0
Event ID:      1309
Task Category: Web Event
Level:         Warning
Keywords:      Classic
User:          N/A
Description:
Event code: 3005
Event message: An unhandled exception has occurred.
Event ID: EventID
Event sequence: 2
Event occurrence: 1
Event detail code: 0
 
Application information:
    Application domain: /LM/W3SVC/2/ROOT/owa-2-DomainID
    Trust level: Full
    Application Virtual Path: /owa
    Application Path: C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\owa\
    Machine name: Exch1
 
Process information:
    Process ID: 21508
    Process name: w3wp.exe
    Account name: NT AUTHORITY\SYSTEM
 
Exception information:
    Exception type: TargetInvocationException
    Exception message: Exception has been thrown by the target of an invocation.
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Owin.Loader.DefaultLoader.<>c__DisplayClass12.<MakeDelegate>b__b(IAppBuilder builder)
   at Owin.Loader.DefaultLoader.<>c__DisplayClass1.<LoadImplementation>b__0(IAppBuilder builder)
   at Microsoft.Owin.Host.SystemWeb.OwinAppContext.Initialize(Action`1 startup)
   at Microsoft.Owin.Host.SystemWeb.OwinBuilder.Build(Action`1 startup)
   at Microsoft.Owin.Host.SystemWeb.OwinHttpModule.InitializeBlueprint()
   at System.Threading.LazyInitializer.EnsureInitializedCore[T](T& target, Boolean& initialized, Object& syncLock, Func`1 valueFactory)
   at Microsoft.Owin.Host.SystemWeb.OwinHttpModule.Init(HttpApplication context)
   at System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr appContext, HttpContext context, MethodInfo[] handlers)
   at System.Web.HttpApplication.InitSpecial(HttpApplicationState state, MethodInfo[] handlers, IntPtr appContext, HttpContext context)
   at System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr appContext, HttpContext context)
   at System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext)
Encryption certificate is absent
   at Microsoft.Exchange.Security.Authentication.Utility.GetCertificates()
   at Microsoft.Exchange.Clients.Owa2.Server.Core.notifications.SignalR.SignalRStartup.Configuration(IAppBuilder app)
  
Request information:
    Request URL: https://mail.contoso.com:URLID/owa/?bO=1
    Request path: /owa/
    User host address: UserHostAddressC:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\owa\
    User: 
    Is authenticated: False
    Authentication Type: 
    Thread account name: NT AUTHORITY\SYSTEM
 
Thread information:
    Thread ID: 24
    Thread account name: NT AUTHORITY\SYSTEM
    Is impersonating: False
    Stack trace:    at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Owin.Loader.DefaultLoader.<>c__DisplayClass12.<MakeDelegate>b__b(IAppBuilder builder)
   at Owin.Loader.DefaultLoader.<>c__DisplayClass1.<LoadImplementation>b__0(IAppBuilder builder)
   at Microsoft.Owin.Host.SystemWeb.OwinAppContext.Initialize(Action`1 startup)
   at Microsoft.Owin.Host.SystemWeb.OwinBuilder.Build(Action`1 startup)
   at Microsoft.Owin.Host.SystemWeb.OwinHttpModule.InitializeBlueprint()
   at System.Threading.LazyInitializer.EnsureInitializedCore[T](T& target, Boolean& initialized, Object& syncLock, Func`1 valueFactory)
   at Microsoft.Owin.Host.SystemWeb.OwinHttpModule.Init(HttpApplication context)
   at System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr appContext, HttpContext context, MethodInfo[] handlers)
   at System.Web.HttpApplication.InitSpecial(HttpApplicationState state, MethodInfo[] handlers, IntPtr appContext, HttpContext context)
   at System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr appContext, HttpContext context)
   at System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext)

Cause


This issue occurs if the Exchange server Auth certificate that's used for OAuth signing is missing from the Exchange server. You can run the following command to check whether the certificate is missing:

Get-ExchangeCertificate (Get-AuthConfig).CurrentCertificateThumbprint

If the certificate is missing, you will receive the following error message:

A special Rpc error occurs on server Exch1: The certificate with thumbprint CertificateValue was not found.
+ CategoryInfo : NotSpecified: (:) [Get-ExchangeCertificate], InvalidOperationException
+ FullyQualifiedErrorId : [Server=Exch1,RequestId=RequestID] [FailureCategory=Cmdlet-InvalidOperationException] CEA009BC,Microsoft.Exchange.Management.SystemConfigurationTasks.GetExchangeCertificate

Resolution


To fix this issue, install Cumulative Update 7 for Exchange Server 2016 or a later cumulative update for Exchange Server 2016.

Workaround


If your organization has multiple Exchange servers, run the following command in the Exchange Management Shell to confirm if the OAuth certificate is present on other Exchange servers:

Get-ExchangeCertificate (Get-AuthConfig).CurrentCertificateThumbprint

If the certificate is present on other Exchange servers, export the certificate and then import it to the Exchange server that has the issue.

If the certificate isn't present on all Exchange servers in the organization, follow these steps to create and deploy a new OAuth certificate to the Exchange server:

  1. Create a new OAuth certificate by running the following command:

    New-ExchangeCertificate -KeySize 2048 -PrivateKeyExportable $true -SubjectName "cn=Microsoft Exchange Server Auth Certificate" -FriendlyName "Microsoft Exchange Server Auth Certificate" -DomainName "contoso.com"

    Note Change the value of the DomainName parameter in the example (contoso.com) to the SMTP domain that's used in your organization.

  2. Set the created certificate to be used for server authentication by running the following commands:

    Set-AuthConfig -NewCertificateThumbprint <ThumbprintFromStep1> -NewCertificateEffectiveDate (Get-Date)
    Set-AuthConfig –PublishCertificate
    Set-AuthConfig -ClearPreviousCertificate

  3. Restart the Microsoft Exchange Service Host Service.

  4. Either run the IISReset command to restart IIS or run the following commands (in elevated mode) to recycle OWA and ECP APP pools:

    Restart-WebAppPool MSExchangeOWAAppPool
    Restart-WebAppPool MSExchangeECPAppPool

    Note In some environments, it may take an hour for the OAuth certificate to be published.

Status


Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

References


Learn about the terminology that Microsoft uses to describe software updates.