Symptoms
In a Microsoft Exchange Server 2010 and Exchange Server 2016 coexistence environment, all Exchange virtual directories URLs point to Exchange Server 2016 (for example, mail.comtoso.com). Services like Autodiscover, Outlook on the web (OWA), Exchange Web Services (EWS) won’t work correctly for users with mailboxes hosted in Exchange Server 2010. Exchange Server 2016 users aren’t affected.
For example, Exchange Server 2010 users trying to sign in to OWA continually receive prompts for credentials, but if all Exchange virtual directories URLs point to Exchange Server 2010, users can sign in to OWA normally.
Additionally, “401,401,ProtocolError” error is logged in Exchange Server 2016 HttpProxy logs.
Cause
The Extended Protection feature is enabled on Exchange Server 2010.
Resolution
Reset the value of Extended Protection and restart the IIS on Exchange Server 2010:
For example:
Set-OWAvirtualdirectory -Server Exch10 -ExtendedProtectionFlags $null -ExtendedProtectionSPNList $null
More information
The Extended Protection feature was introduced by a security update in Windows KB970430 and KB973917 to avoid the Credential relay attack or Man in the middle attack. For more information about the Extended protection feature, see Extended Protection for Authentication Overview.