FIX: Out of memory error when the virtual address space of the SQL Server process is very low in SQL Server

Applies to: SQL Server 2014 Service Pack 2SQL Server 2014 StandardSQL Server 2014 Developer

Symptoms


Assume that you install the 64-bit version of Microsoft SQL Server 2014, 2016 or 2017 on a computer that's running Windows Server 2012 or an earlier version of Windows operating system. When SQL Server uses up all the user-mode virtual address space, you may receive an out of memory error message in the SQL Server error log. Additionally, queries take a long time to finish execution, and the SOS_MEMORY_TOPLEVELBLOCKALLOCATOR wait occurs.

When you examine the following information points, you will find that there is very low available virtual address space:

  • DBCC MEMORYSTATUS - Process/System Counts section - Available Virtual Memory
  • DMV: sys.dm_os_process_memory - column virtual_address_space_available_kb

These values start off around 8 terabytes (TB) on an x64 process and continue to climb down and reach a few gigabytes (GB).

When you are at the stage where the available virtual address space is very low, queries that try to perform memory allocation may also encounter a wait type of CMEMTHREAD.

The following data points will continue to increase over time:

  • DMV: sys.dm_os_process_memory and sys.dm_os_memory_nodes - column virtual_address_space_reserved_kb
  • DBCC MEMORYSTATUS - Memory Manager section - VM Reserved

These values will typically increase in multiples of the "max server memory" value up to almost 8 TB.

Note Related issue is also described in KB 3074434. 

Resolution


This update is included in the following cumulative updates for SQL Server:

      Cumulative Update 4 for SQL Server 2017

      Cumulative update 8 for SQL Server 2016 SP1  

      Cumulative Update 10 for SQL Server 2014 Service Pack 2

Note After you apply this update, you have to add the trace flag 8075 as a startup parameter to enable this change.

Status


Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

References


Learn about the terminology that Microsoft uses to describe software updates.