You cannot specify a NUMA node when you create a process by using the "start" command in Windows Server 2008 R2 or in Windows 7

Symptoms

Consider the following scenario:

• You have a computer that is running Windows Server 2008 R2 or Windows 7.

• You want to use the start /affinity command to create some new processes for a specific preferred NUMA node or for a specific processor group.
  
  Note The start /affinity command is part of the Cmd.exe utility.

In this scenario, the start /affinity command does not let you specify the preferred NUMA node or the processor group. Therefore, you cannot create an affinity between a specific processor and the process when the process is created.

Also, the processes cannot benefit from running on a specially configured set of processors to use the local memory or to distribute processor load when the processes are created.

Notes

• If the a computer has only one processor group, the start /affinity command can create a process that runs on any set of logical processors. However, the command cannot assign the preferred node.

• A computer that has more than 64 active logical processors has more than one processor group.

• The affinity is a tuple that consists of a processor mask and of a processor group number. The processor group number qualifies the processor mask and fully qualifies the processor mask when more than one processor group exists.
  
  For example, the 0x3 processor mask and the 0 processor group represent a different set of processors from the 0x3 processor mask and the 1 processor group.

Cause

This issue occurs because the start command for the Cmd.exe utility of Windows Server 2008 R2 and of Windows 7 does not let a user specify a processor group number.

Notes

• The concept of a processor group was introduced in Windows Server 2008 R2 and in Windows 7 to support more than 64 logical processors.

• A user can specify an affinity mask by using the start /affinity command. However, the operating system randomly assigns the preferred node that dictates the processor group. Therefore, the processor group is not deterministic. Additionally, multiple calls of the start /affinity maskcommand may create processes that are running on different processors even if the same affinity mask is assigned.

Resolution

After you install this hotfix on the computer that is running Windows Server 2008 R2 or Windows 7, you can use /node switch to specify a NUMA node in a start command.

The following are some examples of this new switch in a start command:

start /NODE 1 application1.exe

start /NODE 1 /AFFINITY 0x3 application1.exe

Workaround

To work around this issue, start a process, then change the process affinity.

To change the process affinity, start the Taskmgr.exe utility, click the Processes tab, and then click Set Affinity. Or, use a program that calls the Win32 API.

Status

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

More Information

For the start /affinity command of the Cmd.exe utility, the affinity mask is interpreted differently if you use the /affinity and /node switches together. Specify the affinity mask as if the processor mask of the NUMA node is shifted toward the right side to begin at the bit zero. The process runs on only those processors that are both in the specified affinity mask and in the NUMA node. If no processors are in common, the process runs on only the specified NUMA node.  

With the /node switch, you can create processes that use the local memory of the NUMA-based processors efficiently. For example, assume that you have two processes that communicate to one another frequently by using shared memory. In this scenario, memory latency can be reduced by using the /node switch to create processes that share the same preferred NUMA node.

To create two processes that try to allocate memory from the same NUMA node and that can run on the processors that are outside the specified node, run the following command:

start /NODE 1 application1.exe
start /NODE 1 application2.exe

Note The application1.exe and application2.exe placeholders are for the file name of the executable files for the processes.

To create two processes that are further constrained to run on only some specific processors that are in the same NUMA node, run the following command: 

start /NODE 1 /AFFINITY 0x3 application1.exe
start /NODE 1 /AFFINITY 0xc application2.exe

Note In these commands, application1.exe runs on the two processors that are in the low-order of the node, while application2.exe runs on the next two processors of the node. Additionally, these commands assume that the specified node has at least four logical processors. When the node number is changed to any valid node number for that computer, the affinity mask does not have to be changed.

Together with start /node functionality, the new %HighestNumaNodeNumber% dynamic environment variable is added to the Cmd.exe utility. With this variable, you can check whether a computer has NUMA-based processors, and you can iterate over all the nodes. The following is a sample script that uses this new variable:@echo off rem rem This sample script shows the two features that are added to Cmd.exe in Windows 7 rem Service Pack 1: rem rem %HighestNumaNodeNumber% rem This is a new dynamic environment variable. rem Run 'set /?' for more information. rem rem start /node <NUMA node number> rem /node is a new command-line option for the start command that rem can be used to specify the preferred NUMA node for the process rem that is being started. rem Run 'start /?' for more information. rem rem Start several new processes where each preferred NUMA node of a process is rem assigned in a round robin manner across all the NUMA nodes in the system. rem rem Example: Start 7 new processes where the preferred node is distributed rem across 4 NUMA nodes. rem rem start /node 0 process0 rem start /node 1 process1 rem start /node 2 process2 rem start /node 3 process3 rem start /node 0 process4 rem start /node 1 process5 rem start /node 2 process6 rem rem process0 might be process0.exe or process0.cmd. rem if defined verbose echo on setlocal enableextensions enabledelayedexpansion if "%1"=="" ( echo Usage: %0 ^<number of processes to distribute among NUMA nodes^> goto end ) else ( set ProcessCount=%1 ) rem rem %HighestNumaNodeNumber% is a dynamic environment variable that is available rem starting in Windows 7 Service Pack 1. Make sure that a real environment variable rem by this name is not already defined because that value would be used instead rem of the automatic system-generated value. rem set HighestNumaNodeNumber= if not defined HighestNumaNodeNumber set HighestNumaNodeNumber=3 set /a ProcessCountMinusOne=%ProcessCount% - 1 set /a NumberOfNumaNodes=%HighestNumaNodeNumber% + 1 set start=0 set step=1 set end=%ProcessCountMinusOne% rem rem Round robin the start of each process across the NUMA nodes. rem for /L %%p in (%start%, %step%, %end%) do ( rem rem Note the modulo operator (%) used on the command line must be doubled rem up (%%) when the operator is used in a cmd script. rem set /a node=%%p %% %NumberOfNumaNodes% rem rem Remove 'echo' below to actually start these processes. rem echo start /node !node! process%%p ) :end endlocal

For more information about NUMA support, view the following Microsoft website:

General information about NUMA supportFor more information about software update terminology, click the following article number to view the article in the Microsoft Knowledge Base:

824684 Description of the standard terminology that is used to describe Microsoft software updates