You are intermittently prompted for credentials or experience time-outs when you connect to Authenticated Services

You may experience one or more of the following symptoms. These symptoms may be intermittent or continuous. These symptoms are more likely and more widespread during "high usage" times, such as at the beginning of a business day when increased client load occurs on the servers in the environment.

You may experience the following issues in a web services scenario:

• Web clients receive delayed responses from the web server.
• Web clients are repeatedly prompted for credentials even if the correct credentials are entered.

You may experience the following issues in a web proxy scenario:

• Web clients receive delayed responses from the web server.
• Web clients are repeatedly prompted for credentials even if the correct credentials are entered.

You may experience the following issues in an Exchange client scenario:

• Clients receive delayed responses from the server.
• Clients are repeatedly prompted for credentials even if the correct credentials are entered.

You may experience the following issue in any scenario in which NTLM authentication is used for applications:

You may experience the following issue in a remote file access scenario:You may experience the following issue in any scenario in which Kerberos delegation is being used in a middle-tier service:Notes

• This issue is more likely to occur if one or more of the following conditions are true:
  • There are highly transactional and heavily used services in the environment.
  • There is heavy use of scripts that use the WINNT provider.
  • There are applications and services that are not configured (or are not configurable) to use Kerberos authentication.
  • When the following three conditions are true at the same time:
    • There are many "accounts" domains (in other words, domains that have user accounts in them) in the environment.
    • There are Windows Server 2003-based domain controllers (DCs) .
    • There are applications or services that may authenticate without providing the domain name. For example, there are applications or services that provide <null>\username instead of domainname\username.
• The following symptoms indicate that this issue is occurring in the environment:
  • A Kerberos source event is logged in the System log of application servers. This event indicates that Kerberos PAC validation is failing. The event resembles the following:

    Event Type: Error
    Event Source: Kerberos
    Event Category: None
    Event ID: 7
    User: N/A
    Description: The Kerberos subsystem encountered a PAC verification failure. This indicates that the PAC from the client computer name in realm AD DNS domain name had a PAC which did not verify or was modified. Contact your system administrator.
    Data: 0000: c0000192

  • Text in Netlogon service debug logs (Netlogon.log) matches the text "NlpUserValidateHigher: Can't allocate Client API slot." These entries may appear in any of the Netlogon debug logs of the following servers:
    • The application server
    • The domain controllers in the application servers domain
    • Trusting domain controllers
  • Perfmon performance logging of the Netlogon performance counter for Semaphore Timeouts during the time when the issue is occurring shows numbers greater than zero. This counter value may appear on any of the following servers in this scenario:
    • The application server
    • The domain controllers in the application servers domain
    • Trusting domain controllers

This issue occurs when a high volume of NTLM authentication or Kerberos PAC validation transactions (or both) occur on a Windows-based server, and that volume is greater than the volume that can be handled at one time by the member server or the domain controllers that are providing authentication. In other words, this is caused by an authentication resource bottleneck.

NTLM authentication and PAC validation are performed by dedicated threads in the Lsass.exe process on Windows-based computers. There is a maximum number of these threads that are available to handle these requests at the same time, and if the requests exceed the availability of the threads and the requests cannot wait any longer, this issue occurs.

By default, workstations have one of the threads available for use, and member servers have two of the threads available for use. Domain controllers have one available thread per security channel to trusted domains. This maximum number of threads that are dedicated to this purpose is known as "Maxconcurrentapi" and is configurable.

To resolve the issue, use one or more of the following methods:

• Install the following hotfix, and then follow the steps that are described in the "Registry information" section. After you install this hotfix on Windows Vista, Windows Server 2008, Windows 7, or Windows Server 2008 R2, the maximumlimit of concurrent connections between a client computer and another server or a domain controller for NTLM authentication or PAC validation may be changed up to 150. This should be done on all servers that exhibit Perfmon Netlogon “semaphore time-out” indications in their Performance logs or that have the “NlpUserValidateHigher: Can't allocate Client API slot” text in their Netlogon debug logs.
• For applications and services that are using NTLM, just configure them to use Kerberos authentication instead. The methods to do that will be unique to those applications.
• If Kerberos PAC validation is seen as a symptom, disable Kerberos PAC validation if the service allows this. This should be done on the server that has the Kerberos sourced system event 7’s appearing.
  
  Note Kerberos PAC validation cannot be disabled for IIS application pools or for some Exchange-related services.

Note In order to decide what value to set for the MaxConcurrentApi setting in your environment refer to the Knowledge Base article below.

2688798 How to do performance tuning for NTLM authentication by using the MaxConcurrentApi setting

Hotfix information

A supported hotfix is available from Microsoft. However, this hotfix is intended to correct only the problem that is described in this article. Apply this hotfix only to systems that are experiencing the problem that is described in this article. This hotfix might receive additional testing. Therefore, if you are not severely affected by this problem, we recommend that you wait for the next software update that contains this hotfix.

If the hotfix is available for download, there is a "Hotfix download available" section at the top of this Knowledge Base article. If this section does not appear, contact Microsoft Customer Service and Support to obtain the hotfix.

Note If additional issues occur or if any troubleshooting is required, you might have to create a separate service request. The usual support costs will apply to additional support questions and issues that do not qualify for this specific hotfix. For a complete list of Microsoft Customer Service and Support telephone numbers or to create a separate service request, visit the following Microsoft website: Note The "Hotfix download available" form displays the languages for which the hotfix is available. If you do not see your language, it is because a hotfix is not available for that language.

Prerequisites

To apply this hotfix, your computer must be running Windows 7, Windows Server 2008 R2, Windows Vista Service Pack 2, or Windows Server 2008 Service Pack 2.

Registry information

Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base: After you install the hotfix, increase the Maxconcurrentapi value to a larger number on all servers that have Perfmon Netlogon "semaphore timeout" indications in their Performance logs or that have "NlpUserValidateHigher: Can't allocate Client API slot" text in their Netlogon debug logs. To do this, follow these steps:

1. Start Registry Editor.
2. Locate the following registry subkey:
   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters
3. Create the following registry entry:
   
   Name: MaxConcurrentApi
   Type: REG_DWORD
   Value:Set the value to the larger number, which you tested (any number greater than the default value).
4. At a command prompt, run net stop netlogon, and then run net start netlogon.

Notes

• The maximum value that can be configured depends on the operating system version and whether a hotfix is available.
  • The maximum configurable setting in Windows Server 2003 is 10.
  • The maximum configurable setting in Windows Server 2008 (without the hotfix in this article) is 10. With the hotfix, the maximum is 150.
  • The maximum configurable setting in Windows Server 2008 R2 (without the hotfix in this article) is 10. With the hotfix, the maximum is 150.
• If you decide to increase the MaxConcurrentApivalue to greater than 10, the load and the performance of the desired setting should be tested in a nonproduction environment before you implement in production. This is recommended to make sure that increasing this value does not cause other resource bottlenecks.

To disable Kerberos PAC validation on a server, follow these steps.

1. Start the Regedt32 program.
2. Locate the registry key path HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Kerberos\Parameters.
3. At that registry location, create a DWORD value named ValidateKdcPacSignature.
4. Set the value of ValidateKdcPacSignature value to 0.
5. Restart the server.
   

Note Kerberos PAC validation cannot be disabled for IIS application pools or for some Exchange-related services.

Restart requirement

You must restart the computer after you apply this hotfix.

Hotfix replacement information

This hotfix does not replace a previously released hotfix.

File information

The English (United States) version of this hotfix installs files that have the attributes that are listed in the following tables. The dates and the times for these files are listed in Coordinated Universal Time (UTC). The dates and the times for these files on your local computer are displayed in your local time together with your current daylight saving time (DST) bias. Additionally, the dates and the times may change when you perform certain operations on the files.

• The MANIFEST files (.manifest) and the MUM files (.mum) that are installed for each environment are listed separately in the "Additional file information for Windows Vista and Windows Server 2008" section. MUM and MANIFEST files, and the associated security catalog (.cat) files, are very important to maintaining the state of the updated component. The security catalog files, for which the attributes are not listed, are signed with a Microsoft digital signature.

File information for Windows Vista and for Windows Server 2008

File information for x86-based versions of Windows Server 2008 and of Windows Vista

File information for x64-based versions of Windows Server 2008 and of Windows Vista

File information for IA-64-based versions of Windows Server 2008

File Information for Windows 7 and for Windows Server 2008 R2

Windows 7 and Windows Server 2008 R2 file information notes Important Windows 7 hotfixes and Windows Server 2008 R2 hotfixes are included in the same packages. However, hotfixes on the Hotfix Request page are listed under both operating systems. To request the hotfix package that applies to one or to both operating systems, select the hotfix that is listed under "Windows 7/Windows Server 2008 R2" on the page. Always refer to the "Applies To" section in articles to determine the actual operating system to which each hotfix applies.

• The MANIFEST files (.manifest) and the MUM files (.mum) that are installed for each environment are listed separately in the "Additional file information for Windows Server 2008 R2 and for Windows 7" section. MUM and MANIFEST files, and the associated security catalog (.cat) files, are very important to maintaining the state of the updated component. The security catalog files, for which the attributes are not listed, are signed with a Microsoft digital signature.

For all supported x86-based versions of Windows 7

For all supported x64-based versions of Windows 7 and of Windows Server 2008 R2

For all supported IA-64-based versions of Windows Server 2008 R2

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

This hotfix is included in Windows 7 Service Pack 1 (SP1) and in Windows Server 2008 R2 Service Pack 1 (SP1).

The MaxConcurrentApi setting and the default settings for it are a legacy of Windows 2000 and of the limited hardware capabilities of that time. With older hardware, allowing for additional threads and the RPC traffic they would generate was a serious concern, and there was a possibility of performance bottlenecks if too many threads were created. With newer hardware platforms and improved performance, that hardware performance limitation is less likely to occur. As always, it is important to gauge and understand the performance of the servers in an environment before you increase the potential load by using a high MaxConcurrentApi setting.

For more information about how to use Netlogon service debug logging (Netlogon.log), click the following article number to view the article in the Microsoft Knowledge Base:An additional lessening step can be performed on Windows Server 2003-based domain controllers that have entries in their Netlogon service debug log that indicate that clients are submitting <null>\username instead of domainname\username. The steps are described in the following Microsoft Knowledge Base article:More information about how to use the Netlogon Performance monitoring object is available, together with an update to add that Performance object in Windows Server 2003. There is an update for Windows Server 2003 that lets you monitor the speed and the throughput of NTLM authentications. For more information, click the following article number to view the article in the Microsoft Knowledge Base: For more information about software update terminology, click the following article number to view the article in the Microsoft Knowledge Base:

Additional file information

Additional file information for Windows Vista and for Windows Server 2008

Additional file information for x86-based versions of Windows Vista and of Windows Server 2008

Additional file information for x64-based versions of Windows Server 2008 and of Windows Vista

Additional file information for IA-64-based versions of Windows Server 2008

Additional file information for Windows 7 and for Windows Server 2008 R2

Additional files for all supported x86-based versions of Windows 7

Additional files for all supported x64-based versions of Windows 7 and of Windows Server 2008 R2

Additional files for all supported IA-64-based versions of Windows Server 2008 R2