Event ID 2021 is logged even though lots of non-paged pool memory is available in Windows Server 2003

On a computer that is running Microsoft Windows Server 2003, you run System Monitor to monitor the non-paged pool bytes. System Monitor shows that lots of non-paged pool memory is still available. However, the following event is logged in the System log:

Event ID: 2021
Source: Srv
Description: Server was unable to create a work item %2 times in the last %3 seconds.

Additionally, when you run System Monitor to collect performance data, the following conditions are true for all queues except the Blocking queue:

• The Available Work Items counter of the Server Work Queues object is very low.
• The Queue Length counter of the Server Work Queues object is 0 (zero).
• The Borrowed Work Items counter of the Server Work Queues object is very high.

This problem may occur if the following conditions are true:

• The cache line size of the CPU is less than 0x80.
• The IRPStackSize parameter for the Server service is set to a value that ranges from 33 through 38.
• The load on the Server service increases to a point where the Server service must allocate extra work items.

Service pack information

To resolve this problem, obtain the latest service pack for Windows Server 2003. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

Hotfix information

A supported hotfix is available from Microsoft. However, this hotfix is intended to correct only the problem that is described in this article. Apply this hotfix only to systems that are experiencing this specific problem. This hotfix might receive additional testing. Therefore, if you are not severely affected by this problem, we recommend that you wait for the next software update that contains this hotfix.

If the hotfix is available for download, there is a "Hotfix download available" section at the top of this Knowledge Base article. If this section does not appear, contact Microsoft Customer Service and Support to obtain the hotfix.

Note If additional issues occur or if any troubleshooting is required, you might have to create a separate service request. The usual support costs will apply to additional support questions and issues that do not qualify for this specific hotfix. For a complete list of Microsoft Customer Service and Support telephone numbers or to create a separate service request, visit the following Microsoft Web site: Note The "Hotfix download available" form displays the languages for which the hotfix is available. If you do not see your language, it is because a hotfix is not available for that language.

Prerequisites

To apply this hotfix, you must have Windows Server 2003 Service Pack 1 (SP1) installed on the computer. For more information about how to obtain the latest service pack for Windows Server 2003, click the following article number to view the article in the Microsoft Knowledge Base:

Restart requirement

You must restart the computer after you apply this hotfix.

Hotfix replacement information

This hotfix does not replace any other hotfixes.

File information

The English version of this hotfix has the file attributes (or later file attributes) that are listed in the following table. The dates and times for these files are listed in Coordinated Universal Time (UTC). When you view the file information, it is converted to local time. To find the difference between UTC and local time, use the Time Zone tab in the Date and Time item in Control Panel.

Windows Server 2003, x86-based versions

Windows Server 2003, Itanium-based versions

Windows Server 2003, x64-based versions

Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:

To work around this problem, specify the IRPStackSize parameter for the Server service in the registry. To do this, follow these steps:

1. Click Start, click Run, type regedit, and then click OK.
2. Locate and then click the following registry subkey:
   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters
3. In the right pane, click IRPStackSize.
4. Click Edit, and then click Modify.
5. In the Edit DWORD Value dialog box, click Decimal under Base.
6. In the Data Value box, type a value outside the range of 33 through 38. For example, type 29 or 40.
7. Click OK.

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section. This problem was first corrected in Windows Server 2003 Service Pack 2.

A work item is used to record details of a file I/O request from a remote client. A work item consists of a Server Work Item structure, an I/O request packet (IRP), and a Server Message Block (SMB) header and buffer. The size of the IRP is affected by the IRPStackSize parameter. The size of an SMB header is 0x70 bytes.

When the load on the Server service increases, the system has to allocate extra work items from the non-paged pool. Then, the system has to queue the work items for later processing. To improve performance, the Server service tries to align the SMB header in a work item on a CPU cache line boundary. The Server service also tries to make sure that the SMB header does not cross a page boundary.

However, if the cache line size of the CPU is less than the SMB header size of 0x70 and if the IRPStackSize parameter is very large, a conflict in the work item allocation code may occur. The Server service may align an SMB header on a cache line boundary that is very close to the end of a page. Therefore, the SMB header overlaps the page boundary. This scenario causes low performance. Event ID 2021 is logged when the allocation activity is rejected.

In this scenario, the Server service will acquire a non-paged pool block for a work item. However, the Server service will decide that the non-paged pool block is unusable because of alignment issues. Therefore, the Server service returns the non-paged pool block to the non-paged pool. Meanwhile, the Server service borrows a work item from another server work queue. When the load on the Server service increases, the demand for work items increases. However, the total number of work items remains fixed. Therefore, the Server service experiences low performance because there are not enough work items to accommodate the increased load.

The IRPStackSize parameter specifies the number of stack locations in IRPs that Microsoft Windows 2000 Server and Windows Server 2003 use. For more information about the IRPStackSize parameter, click the following article number to view the article in the Microsoft Knowledge Base:
This problem does not occur on systems whose CPU cache line sizes are 0x80 or more.

This problem occurs on systems that have smaller cache line sizes if any one of the following conditions is true.

On Windows Server 2003-based computers

• The CPU cache line size is 0x40, and the IRPStackSize parameter is decimal 35 or 36.
• The CPU cache line size is 0x20, and the IRPStackSize parameter is decimal 35, 36, or 37.
• The CPU cache line size is 0x10, and the IRPStackSize parameter is decimal 36 or 37.
• The CPU cache line size is 0x08, and the IRPStackSize parameter is decimal 36, 37, or 38.

On Windows Server 2003 Service Pack 1 (SP1)-based computers

• The CPU cache line size is 0x40, and the IRPStackSize parameter is decimal 33 or 34.
• The CPU cache line size is 0x20, and the IRPStackSize parameter is decimal 33, 34, or 35.
• The CPU cache line size is 0x10, and the IRPStackSize parameter is decimal 34 or 35.
• The CPU cache line size is 0x08, and the IRPStackSize parameter is decimal 34 or 35.

Note The small differences between Windows Server 2003 and Windows Server 2003 SP1 are because of the size of the Server Work Item structure in Windows Server 2003 SP1.