FIX: Performance problems occur in NUMA environments during foreign page processing in SQL Server 2012 or SQL Server 2014

Applies to: SQL Server 2012 DeveloperSQL Server 2012 EnterpriseSQL Server 2012 Express More

Symptoms


In NUMA environments, Microsoft SQL Server experiences one or more of the following performance problems:
  • A SQL Server process experiences excessive CPU usage.
  • Queries take a long time to be completed.
  • Applications or SQL Server Agent jobs experience query time-outs or connection time-outs.
  • A SQL Server process is restarted by the Cluster service.
Additionally, you may experience the following symptoms:
  • Elevated values for the SOS_PHYS_PAGE_CACHE wait type when you query the sys.dm_os_wait_stats dynamic management view (DMV)
  • Elevated values for the CMEMTHREAD wait type and SOS_SUSPEND_QUEUE spinlock when you query the sys.dm_os_spinlock_stats DMV

Resolution


Cumulative update information

The issue was first fixed in the following cumulative update of SQL Server.

Cumulative Update 2 for SQL Server 2014

Cumulative Update 1 for SQL Server 2014

Cumulative Update 9 for SQL Server 2012 SP1

Hotfix information

A supported hotfix is available from Microsoft. However, this hotfix is intended to correct only the problem that is described in this article. Apply this hotfix only to systems that are experiencing this specific problem.

If the hotfix is available for download, there is a "Hotfix Download Available" section at the top of this Knowledge Base article. If this section does not appear, submit a request to Microsoft Customer Service and Support to obtain the hotfix.

Note If additional problems occur or if any troubleshooting is required, you might have to create a separate service request. The usual support costs will apply to additional support questions and problems that do not qualify for this specific hotfix. For a complete list of Microsoft Customer Service and Support telephone numbers or to create a separate service request, visit the following Microsoft website: Note The "Hotfix Download Available" form displays the languages for which the hotfix is available. If you do not see your language, it is because a hotfix is not available for that language.


Important This fix is recommended for any SQL Server instance that is installed on a NUMA system, regardless of the SQL Server affinity mask settings. This fix addresses the AWAY processing performance bottleneck in the SQL Server memory manager.
The issue was first fixed in the following cumulative update of SQL Server.

Workaround


To work around these problems, increase the logon and query time-out values for your applications.

Note This workaround may be limited in its effect. We recommend that you apply the hotfix to resolve the problems that are described in the "Symptoms" section.

Status


Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

More Information


When Microsoft SQL Server’s memory manager allocates memory for a node in non-uniform memory access (NUMA)-enabled environments, Windows can allocate memory from both the local and remote nodes (referred to as "away" or "foreign" pages) until the desired target for each node is reached. Since these "Away pages" are not ideal from a performance perspective, SQL Server process tries to assign the away buffers to the proper memory node or release these pages back to operating system. This "AWAY processing" can be CPU intensive and lead to a reduction in batch rates during a period of increased CPU usage.

References


For more information about how to understand the concept of foreign pages, go to the following MSDN blog post website: