Symptoms

You may experience one or more of the following symptoms. These symptoms may be intermittent or continuous. These symptoms are more likely and more widespread during "high usage" times, such as at the beginning of a business day when increased client load occurs on the servers in the environment.

You may experience the following issues in a web services scenario:

  • Web clients receive delayed responses from the web server.

  • Web clients are repeatedly prompted for credentials even if the correct credentials are entered.

You may experience the following issues in a web proxy scenario:

  • Web clients receive delayed responses from the web server.

  • Web clients are repeatedly prompted for credentials even if the correct credentials are entered.

You may experience the following issues in an Exchange client scenario:

  • Clients receive delayed responses from the server.

  • Clients are repeatedly prompted for credentials even if the correct credentials are entered.

You may experience the following issue in any scenario in which NTLM authentication is used for applications:

Line of business or custom applications that use NTLM authentication fail. Additionally, you may receive different errors that are intermittent and may include "access denied."You may experience the following issue in a remote file access scenario:

Windows clients receive "access denied" errors or delayed responses from the file server.You may experience the following issue in any scenario in which Kerberos delegation is being used in a middle-tier service:

The clients gain access successfully at first but then lose access to the same resources. Additionally, you may be repeatedly prompted for credentials or experience "access denied" errors.Notes

  • This issue is more likely to occur if one or more of the following conditions are true:

    • There are highly transactional and heavily used services in the environment.

    • There is heavy use of scripts that use the WINNT provider.

    • There are applications and services that are not configured (or are not configurable) to use Kerberos authentication.

    • When the following three conditions are true at the same time:

      • There are many "accounts" domains (in other words, domains that have user accounts in them) in the environment.

      • There are Windows Server 2003-based domain controllers (DCs) .

      • There are applications or services that may authenticate without providing the domain name. For example, there are applications or services that provide <null>\username instead of domainname\username.

  • The following symptoms indicate that this issue is occurring in the environment:

    • A Kerberos source event is logged in the System log of application servers. This event indicates that Kerberos PAC validation is failing. The event resembles the following:

    • Text in Netlogon service debug logs (Netlogon.log) matches the text "NlpUserValidateHigher: Can't allocate Client API slot." These entries may appear in any of the Netlogon debug logs of the following servers:

      • The application server

      • The domain controllers in the application servers domain

      • Trusting domain controllers

    • Perfmon performance logging of the Netlogon performance counter for Semaphore Timeouts during the time when the issue is occurring shows numbers greater than zero. This counter value may appear on any of the following servers in this scenario:

      • The application server

      • The domain controllers in the application servers domain

      • Trusting domain controllers

Cause

This issue occurs when a high volume of NTLM authentication or Kerberos PAC validation transactions (or both) occur on a Windows-based server, and that volume is greater than the volume that can be handled at one time by the member server or the domain controllers that are providing authentication. In other words, this is caused by an authentication resource bottleneck.

NTLM authentication and PAC validation are performed by dedicated threads in the Lsass.exe process on Windows-based computers. There is a maximum number of these threads that are available to handle these requests at the same time, and if the requests exceed the availability of the threads and the requests cannot wait any longer, this issue occurs.

By default, workstations have one of the threads available for use, and member servers have two of the threads available for use. Domain controllers have one available thread per security channel to trusted domains. This maximum number of threads that are dedicated to this purpose is known as "Maxconcurrentapi" and is configurable.

Resolution

To resolve the issue, use one or more of the following methods:

  • Install the following hotfix, and then follow the steps that are described in the "Registry information" section. After you install this hotfix on Windows Vista, Windows Server 2008, Windows 7, or Windows Server 2008 R2, the maximumlimit of concurrent connections between a client computer and another server or a domain controller for NTLM authentication or PAC validation may be changed up to 150. This should be done on all servers that exhibit Perfmon Netlogon “semaphore time-out” indications in their Performance logs or that have the “NlpUserValidateHigher: Can't allocate Client API slot” text in their Netlogon debug logs.

  • For applications and services that are using NTLM, just configure them to use Kerberos authentication instead. The methods to do that will be unique to those applications.

Note In order to decide what value to set for the MaxConcurrentApi setting in your environment refer to the Knowledge Base article below.

2688798 How to do performance tuning for NTLM authentication by using the MaxConcurrentApi setting

Hotfix information

A supported hotfix is available from Microsoft. However, this hotfix is intended to correct only the problem that is described in this article. Apply this hotfix only to systems that are experiencing the problem that is described in this article. This hotfix might receive additional testing. Therefore, if you are not severely affected by this problem, we recommend that you wait for the next software update that contains this hotfix.

If the hotfix is available for download, there is a "Hotfix download available" section at the top of this Knowledge Base article. If this section does not appear, contact Microsoft Customer Service and Support to obtain the hotfix.

Note If additional issues occur or if any troubleshooting is required, you might have to create a separate service request. The usual support costs will apply to additional support questions and issues that do not qualify for this specific hotfix. For a complete list of Microsoft Customer Service and Support telephone numbers or to create a separate service request, visit the following Microsoft website:

http://support.microsoft.com/contactus/?ws=supportNote The "Hotfix download available" form displays the languages for which the hotfix is available. If you do not see your language, it is because a hotfix is not available for that language.

Prerequisites

To apply this hotfix, your computer must be running Windows 7, Windows Server 2008 R2, Windows Vista Service Pack 2, or Windows Server 2008 Service Pack 2.

Registry information

Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:

322756 How to back up and restore the registry in WindowsAfter you install the hotfix, increase the Maxconcurrentapi value to a larger number on all servers that have Perfmon Netlogon "semaphore timeout" indications in their Performance logs or that have "NlpUserValidateHigher: Can't allocate Client API slot" text in their Netlogon debug logs. To do this, follow these steps:

  1. Start Registry Editor.

  2. Locate the following registry subkey:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters

  3. Create the following registry entry:

    Name: MaxConcurrentApi
    Type: REG_DWORD
    Value:Set the value to the larger number, which you tested (any number greater than the default value).

  4. At a command prompt, run net stop netlogon, and then run net start netlogon.

Notes

  • The maximum value that can be configured depends on the operating system version and whether a hotfix is available.

    • The maximum configurable setting in Windows Server 2003 is 10.

    • The maximum configurable setting in Windows Server 2008 (without the hotfix in this article) is 10. With the hotfix, the maximum is 150.

    • The maximum configurable setting in Windows Server 2008 R2 (without the hotfix in this article) is 10. With the hotfix, the maximum is 150.

  • If you decide to increase the MaxConcurrentApivalue to greater than 10, the load and the performance of the desired setting should be tested in a nonproduction environment before you implement in production. This is recommended to make sure that increasing this value does not cause other resource bottlenecks.

Restart requirement

You must restart the computer after you apply this hotfix.

Hotfix replacement information

This hotfix does not replace a previously released hotfix.

File information

The English (United States) version of this hotfix installs files that have the attributes that are listed in the following tables. The dates and the times for these files are listed in Coordinated Universal Time (UTC). The dates and the times for these files on your local computer are displayed in your local time together with your current daylight saving time (DST) bias. Additionally, the dates and the times may change when you perform certain operations on the files.

  • The MANIFEST files (.manifest) and the MUM files (.mum) that are installed for each environment are listed separately in the "Additional file information for Windows Vista and Windows Server 2008" section. MUM and MANIFEST files, and the associated security catalog (.cat) files, are very important to maintaining the state of the updated component. The security catalog files, for which the attributes are not listed, are signed with a Microsoft digital signature.

File information for Windows Vista and for Windows Server 2008

File information for x86-based versions of Windows Server 2008 and of Windows Vista

File name

File version

File size

Date

Time

Platform

Netlogon.dll

6.0.6002.22289

592,896

16-Dec-2009

12:09

x86

Nlsvc.mof

Not Applicable

2,873

03-Apr-2009

21:24

Not Applicable

File information for x64-based versions of Windows Server 2008 and of Windows Vista

File name

File version

File size

Date

Time

Platform

Netlogon.dll

6.0.6002.22289

716,800

16-Dec-2009

12:07

x64

Nlsvc.mof

Not Applicable

2,873

03-Apr-2009

20:58

Not Applicable

File information for IA-64-based versions of Windows Server 2008

File name

File version

File size

Date

Time

Platform

Netlogon.dll

6.0.6002.22289

1,216,512

16-Dec-2009

12:05

IA-64

Nlsvc.mof

Not Applicable

2,873

03-Apr-2009

20:59

Not Applicable

File Information for Windows 7 and for Windows Server 2008 R2

Windows 7 and Windows Server 2008 R2 file information notes
Important Windows 7 hotfixes and Windows Server 2008 R2 hotfixes are included in the same packages. However, hotfixes on the Hotfix Request page are listed under both operating systems. To request the hotfix package that applies to one or to both operating systems, select the hotfix that is listed under "Windows 7/Windows Server 2008 R2" on the page. Always refer to the "Applies To" section in articles to determine the actual operating system to which each hotfix applies.

  • The MANIFEST files (.manifest) and the MUM files (.mum) that are installed for each environment are listed separately in the "Additional file information for Windows Server 2008 R2 and for Windows 7" section. MUM and MANIFEST files, and the associated security catalog (.cat) files, are very important to maintaining the state of the updated component. The security catalog files, for which the attributes are not listed, are signed with a Microsoft digital signature.

For all supported x86-based versions of Windows 7

File name

File version

File size

Date

Time

Platform

Netlogon.dll

6.1.7600.20576

563,712

16-Nov-2009

06:40

x86

Nlsvc.mof

Not Applicable

2,873

10-Jun-2009

21:29

Not Applicable

For all supported x64-based versions of Windows 7 and of Windows Server 2008 R2

File name

File version

File size

Date

Time

Platform

Netlogon.dll

6.1.7600.20576

692,736

16-Nov-2009

07:45

x64

Nlsvc.mof

Not Applicable

2,873

10-Jun-2009

20:47

Not Applicable

For all supported IA-64-based versions of Windows Server 2008 R2

File name

File version

File size

Date

Time

Platform

Netlogon.dll

6.1.7600.20576

1,148,416

16-Nov-2009

06:10

IA-64

Nlsvc.mof

Not Applicable

2,873

10-Jun-2009

20:52

Not Applicable



Status

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

More Information

This hotfix is included in Windows 7 Service Pack 1 (SP1) and in Windows Server 2008 R2 Service Pack 1 (SP1).

The MaxConcurrentApi setting and the default settings for it are a legacy of Windows 2000 and of the limited hardware capabilities of that time. With older hardware, allowing for additional threads and the RPC traffic they would generate was a serious concern, and there was a possibility of performance bottlenecks if too many threads were created. With newer hardware platforms and improved performance, that hardware performance limitation is less likely to occur. As always, it is important to gauge and understand the performance of the servers in an environment before you increase the potential load by using a high MaxConcurrentApi setting.

For more information about how to use Netlogon service debug logging (Netlogon.log), click the following article number to view the article in the Microsoft Knowledge Base:

109626 Enabling debug logging for the Net Logon service An additional lessening step can be performed on Windows Server 2003-based domain controllers that have entries in their Netlogon service debug log that indicate that clients are submitting <null>\username instead of domainname\username. The steps are described in the following Microsoft Knowledge Base article:

923241 The Lsass.exe process may stop responding if you have many external trusts on a Windows Server 2003-based domain controller More information about how to use the Netlogon Performance monitoring object is available, together with an update to add that Performance object in Windows Server 2003. There is an update for Windows Server 2003 that lets you monitor the speed and the throughput of NTLM authentications. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

928576 New performance counters for Windows Server 2003 let you monitor the performance of Netlogon authentication

There is an update for Windows Server 2008 R2 that introduces new events to track Netlogoan API overload:

New event log entries that track NTLM authentication delays and failures in Windows Server 2008 R2 are available
http://support.microsoft.com/default.aspx?scid=kb;EN-US;2654097
For more information about software update terminology, click the following article number to view the article in the Microsoft Knowledge Base:

824684

Description of the standard terminology that is used to describe Microsoft software updates

Additional file information

Additional file information for Windows Vista and for Windows Server 2008

Additional file information for x86-based versions of Windows Vista and of Windows Server 2008

File name

Update.mum

File version

Not Applicable

File size

3,068

Date (UTC)

16-Dec-2009

Time (UTC)

21:16

Platform

Not Applicable

File name

X86_d097c3e62c5fe28649de747cfa96d8cc_31bf3856ad364e35_6.0.6002.22289_none_8f4d35d9370e9c53.manifest

File version

Not Applicable

File size

705

Date (UTC)

16-Dec-2009

Time (UTC)

21:16

Platform

Not Applicable

File name

X86_microsoft-windows-security-netlogon_31bf3856ad364e35_6.0.6002.22289_none_ffda50c84e769578.manifest

File version

Not Applicable

File size

22,701

Date (UTC)

16-Dec-2009

Time (UTC)

14:05

Platform

Not Applicable

Additional file information for x64-based versions of Windows Server 2008 and of Windows Vista

File name

Amd64_0fe0181b07108c9de21c48d7ff24c52a_31bf3856ad364e35_6.0.6002.22289_none_f87d1e5f85e49530.manifest

File version

Not Applicable

File size

1,060

Date (UTC)

16-Dec-2009

Time (UTC)

21:16

Platform

Not Applicable

File name

Amd64_microsoft-windows-security-netlogon_31bf3856ad364e35_6.0.6002.22289_none_5bf8ec4c06d406ae.manifest

File version

Not Applicable

File size

23,180

Date (UTC)

16-Dec-2009

Time (UTC)

15:52

Platform

Not Applicable

File name

Update.mum

File version

Not Applicable

File size

3,092

Date (UTC)

16-Dec-2009

Time (UTC)

21:16

Platform

Not Applicable

File name

Wow64_microsoft-windows-security-netlogon_31bf3856ad364e35_6.0.6002.22289_none_664d969e3b34c8a9.manifest

File version

Not Applicable

File size

18,332

Date (UTC)

16-Dec-2009

Time (UTC)

14:00

Platform

Not Applicable

Additional file information for IA-64-based versions of Windows Server 2008

File name

Ia64_93a1c37632c96e8463a7fa3608662f92_31bf3856ad364e35_6.0.6002.22289_none_f505daf555102feb.manifest

File version

Not Applicable

File size

1,058

Date (UTC)

16-Dec-2009

Time (UTC)

21:16

Platform

Not Applicable

File name

Ia64_microsoft-windows-security-netlogon_31bf3856ad364e35_6.0.6002.22289_none_ffdbf4be4e749e74.manifest

File version

Not Applicable

File size

23,156

Date (UTC)

16-Dec-2009

Time (UTC)

16:08

Platform

Not Applicable

File name

Update.mum

File version

Not Applicable

File size

2,247

Date (UTC)

16-Dec-2009

Time (UTC)

21:16

Platform

Not Applicable

File name

Wow64_microsoft-windows-security-netlogon_31bf3856ad364e35_6.0.6002.22289_none_664d969e3b34c8a9.manifest

File version

Not Applicable

File size

18,332

Date (UTC)

16-Dec-2009

Time (UTC)

14:00

Platform

Not Applicable

Additional file information for Windows 7 and for Windows Server 2008 R2

Additional files for all supported x86-based versions of Windows 7




File name

File version

File size

Date

Time

Platform

Package_for_kb975363_rtm~31bf3856ad364e35~x86~~6.1.1.0.mum

Not Applicable

1,947

16-Nov-2009

09:45

Not Applicable

X86_microsoft-windows-security-netlogon_31bf3856ad364e35_6.1.7600.20576_none_fe237c4db262181f.manifest

Not Applicable

35,541

16-Nov-2009

08:08

Not Applicable

Additional files for all supported x64-based versions of Windows 7 and of Windows Server 2008 R2


File name

File version

File size

Date

Time

Platform

Amd64_microsoft-windows-security-netlogon_31bf3856ad364e35_6.1.7600.20576_none_5a4217d16abf8955.manifest

Not Applicable

35,547

16-Nov-2009

08:11

Not Applicable

Package_for_kb975363_rtm~31bf3856ad364e35~amd64~~6.1.1.0.mum

Not Applicable

2,181

16-Nov-2009

09:45

Not Applicable

Wow64_microsoft-windows-security-netlogon_31bf3856ad364e35_6.1.7600.20576_none_6496c2239f204b50.manifest

Not Applicable

16,596

16-Nov-2009

08:01

Not Applicable

Additional files for all supported IA-64-based versions of Windows Server 2008 R2


File name

File version

File size

Date

Time

Platform

Ia64_microsoft-windows-security-netlogon_31bf3856ad364e35_6.1.7600.20576_none_fe252043b260211b.manifest

Not Applicable

35,544

16-Nov-2009

09:06

Not Applicable

Package_for_kb975363_rtm~31bf3856ad364e35~ia64~~6.1.1.0.mum

Not Applicable

1,683

16-Nov-2009

09:45

Not Applicable

Wow64_microsoft-windows-security-netlogon_31bf3856ad364e35_6.1.7600.20576_none_6496c2239f204b50.manifest

Not Applicable

16,596

16-Nov-2009

08:01

Not Applicable

Need more help?

Expand your skills
Explore Training
Get new features first
Join Microsoft Insiders

Was this information helpful?

What affected your experience?

Thank you for your feedback!

×