Symptoms
In NUMA environments, Microsoft SQL Server experiences one or more of the following performance problems:
-
A SQL Server process experiences excessive CPU usage.
-
Queries take a long time to be completed.
-
Applications or SQL Server Agent jobs experience query time-outs or connection time-outs.
-
A SQL Server process is restarted by the Cluster service.
Additionally, you may experience the following symptoms:
-
Elevated values for the SOS_PHYS_PAGE_CACHE wait type when you query the sys.dm_os_wait_stats dynamic management view (DMV)
-
Elevated values for the CMEMTHREAD wait type and SOS_SUSPEND_QUEUE spinlock when you query the sys.dm_os_spinlock_stats DMV
Resolution
Cumulative update information
The issue was first fixed in the following cumulative update of SQL Server.
Cumulative Update 2 for SQL Server 2014 /en-us/help/2967546
Cumulative Update 1 for SQL Server 2014 /en-us/help/2931693
Cumulative Update 9 for SQL Server 2012 SP1 /en-us/help/2931078
Each new cumulative update for SQL Server contains all the hotfixes and all the security fixes that were included with the previous cumulative update. Check out the latest cumulative updates for SQL Server:
Hotfix information
A supported hotfix is available from Microsoft. However, this hotfix is intended to correct only the problem that is described in this article. Apply this hotfix only to systems that are experiencing this specific problem.
If the hotfix is available for download, there is a "Hotfix Download Available" section at the top of this Knowledge Base article. If this section does not appear, submit a request to Microsoft Customer Service and Support to obtain the hotfix.
Note If additional problems occur or if any troubleshooting is required, you might have to create a separate service request. The usual support costs will apply to additional support questions and problems that do not qualify for this specific hotfix. For a complete list of Microsoft Customer Service and Support telephone numbers or to create a separate service request, visit the following Microsoft website:
http://support.microsoft.com/contactus/?ws=supportNote The "Hotfix Download Available" form displays the languages for which the hotfix is available. If you do not see your language, it is because a hotfix is not available for that language.
Important This fix is recommended for any SQL Server instance that is installed on a NUMA system, regardless of the SQL Server affinity mask settings. This fix addresses the AWAY processing performance bottleneck in the SQL Server memory manager.
The issue was first fixed in the following cumulative update of SQL Server.
Workaround
To work around these problems, increase the logon and query time-out values for your applications.
Note This workaround may be limited in its effect. We recommend that you apply the hotfix to resolve the problems that are described in the "Symptoms" section.
Status
Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.
More Information
When Microsoft SQL Server’s memory manager allocates memory for a node in non-uniform memory access (NUMA)-enabled environments, Windows can allocate memory from both the local and remote nodes (referred to as "away" or "foreign" pages) until the desired target for each node is reached. Since these "Away pages" are not ideal from a performance perspective, SQL Server process tries to assign the away buffers to the proper memory node or release these pages back to operating system. This "AWAY processing" can be CPU intensive and lead to a reduction in batch rates during a period of increased CPU usage.
References
For more information about how to understand the concept of foreign pages, go to the following MSDN blog post website:
How It Works: SQL Server (NUMA Local, Foreign and Away Memory Blocks)
