823440 Use of the /3GB switch in Exchange Server 2003 on a Windows Server 2003-based system
810371 Using the /Userva switch on Windows Server 2003-based computers that are running Exchange Server
If you have 1 gigabyte (GB) or more of physical memory (RAM) installed on a server that is running Microsoft Exchange Server 2003, you must make sure that Exchange 2003 can make efficient use of that memory.
Note Exchange 2003 performs an optimal memory configuration check when the store process starts. If the memory settings are not optimal, event 9665 is logged in Event Viewer. This message is logged if one or more of the following conditions are true:
- The server is running any edition of Microsoft Windows 2000 Server and the SystemPages value in the registry is set outside the range of 24000 to 31000.
Note The SystemPages subkey is in the following location:HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\SystemPages
- The server is running Microsoft Windows 2000 Advanced Server or Microsoft Windows 2000 Datacenter Server, has 1 GB or more of physical memory, but does not have the /3GB switch set in the boot.ini file.
- The server is running Windows Server 2003 Standard, Windows Server 2003 Enterprise or Windows Server 2003 Datacenter edition, and the SystemPages value in the registry is set to something other than 0.
- The server is running Windows Server 2003 Standard Edition, Windows Server 2003 Enterprise Edition or Windows Server 2003 Datacenter Edition, has 1 GB or more of physical memory and the /3GB switch is set. However, the /USERVA setting is not present in the boot.ini file.
- The server is running any edition of Microsoft Windows 2000 Server or Windows Server 2003, and the HeapDeCommitFreeBlockThreshold value in the registry is set to something other than 0x00040000.
Note 0x00040000 is a hexadecimal value. You should type 40000 (Hexadecimal) in the registry for the HeapDeCommitFreeBlockThreshold value.
The memory configuration check does not occur on servers that are running Microsoft Small Business Server. When the 9665 event is raised, a DWORD is written to the event log as additional data.
If you want to understand which settings are not set correctly, then running an Exchange Best Practices Analyzer tool health check will report what settings must be changed. For more information, visit the following Microsoft website:
Parameter: Suppress Memory Configuration Notification
Warning We do not recommend this configuration as it could lead to server degradation as it hides potential warning events from administrators.
- For Exchange Server computers that do not contain any mailboxes or public folders, such as mail gateways, we do not recommend setting the /3GB switch in boot.ini, independently of installed physical memory size.
- For Exchange Server computers which are at the same time Active Directory Domain Controllers or global catalog servers we also do not recommend setting the /3GB switch in boot.ini. We recommend having dedicated Active Directory Domain Controllers or global catalog servers.
- Because Microsoft Small Business Server installs as an integrated domain controller and Exchange server, we also do not recommend setting the /3GB switch when you run Exchange Server 2003 in a Small Business Server environment. However, the other concepts and settings that are described in this article apply equally to Exchange Server 2003 in a Microsoft Small Business Server environment.
The following sections in this article contain recommendations for each of these settings.
Things to check on Windows 2000 Server/Exchange Server 2000
- /3GB is enabled in the boot.ini file (on all versions except Windows 2000 Standard Edition)
- SystemPages setting is configured in the registry
- /BaseVideo switch is added to the boot.ini file
Things to check on Windows Server 2003/Exchange Server 2003
- Verify Memory\System Free Page Table Entries are greater than 12,000
- /3GB is enabled in the boot.ini file
- /USERVA has been added in the boot.ini file
- /BaseVideo switch is added to the boot.ini file
Virtual address space concepts
Independent of the hardware configuration, the number of databases, and the number of users on the server, the Exchange Information Store (Store.exe) process in Exchange 2003 has a finite amount of memory that it can address. This amount is known as the virtual address space. In most scenarios, the usage of this virtual address space for the information store dictates the overall performance and scalability of the Exchange Server 2003 mailbox servers. For small to medium-sized servers, Exchange Server 2003 automatically makes the best balance. However, for larger servers that you may want to manually adjust some tuning parameters.
If the Exchange Server 2003 computer has 1 gigabyte (GB) of memory or more installed and if the computer is home to mailboxes or public folders, make sure that you add the /3GB switch to the Boot.ini file on the server. If the server has no mailboxes or public folders on it, such as a mail gateway, we do not recommend that you use the /3GB switch. By default, the Microsoft Windows 2000 Advanced Server and Windows Server 2003 operating systems reserve 2 GB of virtual address space for Kernel mode usage, and 2 GB for User mode. Virtual address space for a specific process is allocated at startup and increases as more memory is used during operation. Typically, the actual memory usage (the working set) of a process is much less than the address space allocated to that process. On a computer that is running Exchange Server 2003 with 1 GB or more of memory, you must modify the Windows 2000 Advanced Server and Windows Server 2003 operating systems so that 3 GB of User mode memory space is available. You can do this by using the /3GB switch in the Boot.ini file.
Using this switch reduces the memory available in the following system pools:
- Nonpaged Pool
- Paged Pool
- System Page Table Entries (PTEs)
If the memory reduction in the pools is too great in a specific server installation, the server or the applications may generate an error or seem to stop responding.
For example, an Exchange server that has 2 GB of physical RAM that does not use the /3GB switch in the Boot.ini file will run out of memory when the Store.exe virtual address space reaches 2 GB. The Windows Task Manager shows that only about 1.5 GB is actually being used. However, the server will, in fact, be out of memory. You can monitor the virtual address consumption with performance monitoring. Add the Virtual Bytes counter for the Store.exe process to make sure of an accurate reading of the virtual space. The Store.exe process is the only Exchange process that you must monitor. Other Exchange processes do not grow large enough to cause any problems.
In addition to modifying the /3GB switch on a Windows 2000 Advanced Server-based computer or Windows Server 2003 computer, you must also configure the SystemPages registry in the following registry subkey:
Do not set the /3GB switch if you are running Windows 2000 Server Standard Edition. This memory tuning switch is not supported on Windows 2000 Server Standard Edition. Although the server will not generate an error message if you do this, the result of setting this switch is that a false memory address space will exist. When a process tries to access this higher address space, a Stop error message on a Stop error occurs, and the server stops responding.
Note The /3GB tuning switch is supported on all releases of Windows Server 2003 including Windows Server 2003 Standard Edition. For more information about how to set the /3GB switch, click the following article number to view the article in the Microsoft Knowledge Base:
The /USERVA switch is new to Windows Server 2003 and provides better detail for splitting memory allocations between user mode and kernel mode. This behavior lets you scale the server to a larger number of users without the risk of exhausting system resources. With the /USERVA switch, you can customize how the memory is allocated when you use the /3GB switch. The number following /USERVA= is how much memory in megabytes (MB) that will be allocated to each process. If you set /3GB /USERVA=3030, this reserves 3,030 MB of memory to the process space, compared to 3,072 MB when you use the /3GB switch alone. The 42 MB that is saved when you set /USERVA=3030 is used to increase the kernel memory space, free system page table entries (PTEs). The PTE memory pool is increased by the difference between 3 GB (specified by the /3GB switch) and the value that is assigned to the /USERVA switch.
It is best that you configure all Windows Server 2003-based servers that run Exchange and are configured by using the /3GB switch to also use the /USERVA=3030 switch to enable more overall system page table entries (PTE) on the server. After you install a Windows Server 2003-based server, you must modify the Boot.ini file to add the /3GB and /USERVA=3030 parameters to the startup line. For example:
multi(0)disk(0)rdisk(0)partition(2)\WINNT="Microsoft Windows Server 2003" /fastdetect /3GB /USERVA=3030
Monitoring Free System Page Table Entries
You can view low PTEs directly by using Performance Monitor. Look for the object Free System Page Table Entries under the Memory counter. Values that are less than 8,000 are too low and may cause server instability issues. Therefore, the /USERVA switch must be adjusted down to increase the overall amount of PTEs on the server. The target value for Free System PTEs is 12,000 or larger.
When virtual memory in the Store.exe process runs low, the performance of the Exchange Server 2003 server can decrease significantly. When the largest free block of virtual memory is reduced to 32 MB, an Event ID 9582 Warning event is generated in the Application log of the Event Viewer. When you see this event, it is best to restart the Store.exe process at the next available opportunity. If the largest block of memory is reduced even more, to the 16 MB level, an Event ID 9582 Error event is generated in the Application log of the Event Viewer. If this event occurs, the server is approaching a critical operating condition and must be restarted at the next opportunity. After this error occurs, memory may become exhausted in several hours. If you do not respond to these events, you may experience one or more of the following symptoms:
- Intermittent message delivery failures.
- Intermittent failure of the Internet mail (IMAIL) conversion process that is indicated by Event ID 12800.
- Certain Exchange-aware antivirus programs may generate error messages or events.
Removing unnecessary video drivers
An Exchange server will not typically require a powerful graphics card or driver. You should make sure that the only the default VGA driver is installed on the server to help increase the PTEs that are available on any given server. You should also add the /BASEVIDEO switch to the Boot.ini startup parameters. This will also force windows to use a less resource-intensive default VGA driver. This is found to save around 1000 System PTEs or more.
In addition to switching to the default VGA driver, we also recommend to remove any graphics card accelerator drivers that may have been installed with the original video card driver. These additional drivers require valuable resources that could reduce the memory available to system pools of the operating system.
Maximizing virtual address space
It is best to monitor the virtual address space for the Information Store on large mailbox servers to make sure that performance and stability does not decrease. The easiest way to do this is to monitor the VM Largest Block Size counter of the MSExchangeIS performance object in the Performance utility. This value is shown in bytes. After you start the computer, it is typical to see a sharp decrease in the largest virtual memory block size. However, after a day or two of operation, the value approaches a typical operating level. A value for the largest free virtual memory block that is more than 200,000,000 bytes (about 200 MB) indicates a healthy server. If you notice a reduced value than this, monitor the server more closely. If you experience a low virtual address space:
- If you have 1 GB or more of physical memory installed, make sure that the /3GB switch is set in the Boot.ini file if the following conditions are true on the server:
- The server is home to mailboxes or public folders.
- The operating system is Windows 2000 Advanced Server or Windows Server 2003.
If the server has no mailboxes or public folders on it, such as a mail gateway, we do not recommend that you use the /3GB switch.
- If Exchange Server 2003 is installed on a Windows Server 2003-based server, and if it has the /3GB switch set, make sure that the /USERVA=3030 switch is set in the Boot.ini file. If Exchange Server 2003 is installed on a Windows 2000-based server, make sure that the server is running Windows 2000 Service Pack 3 (SP3) or a later version.
- If you have 1 GB or more physical memory installed, configure the HeapDeCommitFreeBlockThreshold registry value in the following registry key, and then restart the server:HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session ManagerFor more information about how to use and configure this registry value, click the following article number to view the article in the Microsoft Knowledge Base:
Value name: HeapDeCommitFreeBlockThreshold
Value type: REG_DWORD
Value data: 0x00040000 (recommended)
Value default: not present315407 The "HeapDecommitFreeBlockThreshold" registry key
Store Database Cache size
The Store Database Cache is also known as the ESE buffer, and it provides a large caching area for database transactions before they are committed to the store. By default, Exchange Server 2003 queries the memory configuration of the local computer, and then allocates 896 MB of RAM if the /3GB switch is set in the Boot.ini file, or 576 MB of RAM if the /3GB switch is not set. Where a server is heavily loaded or where disk performance is not optimal, a large ESE buffer increases overall system performance. Depending on your configuration, you may have to increase or reduce the size of this buffer to obtain the best overall performance.
In a scenario where Exchange Server 2003 is used in an environment where it co-exists with other server-side programs, it may monopolize the available memory resources. The Dynamic Buffer Allocation (DBA) algorithm is responsible for returning memory to the operating system if other programs require it. However, you can manually limit the memory that Exchange Server 2003 uses by reducing the ESE buffer.
Warning On servers that have more than 2 GB of memory, it may help increase the size of the ESE buffer. Because of virtual address space limitations, this value must not be set greater than 1,200 MB.
Before you increase the maximum buffer size, we recommend that you use the Windows Performance utility to monitor the memory of the server under a typical load. To do this, monitor the following performance object and value:
Performance counter: Virtual Bytes
For example, if you have a server that is configured to use the /3GB switch in the Boot.ini file, and performance monitoring shows the virtual bytes count at 2.5 GB when the server is heavily loaded, you may be able to increase your maximum buffer size by about 300 MB, for a total size of 1,200 MB.
Increasing the buffer size may adversely affect server performance. A larger buffer means that more virtual address space is consumed. Therefore, if the server experiences virtual memory address space constraints, increasing the buffer size may create operating system instability. This could lead to an unresponsive server. On a very large mailbox server, you may have to decrease the default buffer size to prevent system instability.
How to modify the ESE buffer size
The msExchESEParamCacheSizeMax parameter controls the ESE buffer size. Its value is expressed as a page count, and must be set to an exact multiple of 8192 for maximum efficiency:
- The default size on servers with the /3GB switch set: 229376 (896 MB)
- The default size on servers without the /3GB switch set: 147456 (576 MB)
- The recommended maximum value with the /3GB switch set: 311296 (1.2 GB)
Note You should first monitor the virtual bytes for the store process as stated earlier in this article before you increase this value over the default.
- The recommended maximum value without the /3GB switch set: 196608 (768 MB)
Note Youshould first monitor the virtual bytes for the store process as stated earlier in this article before you increase this value over the default.
Note The replay of transaction logs is significantly faster when the ESE buffer is set to a large size. You may want to temporarily increase the ESE buffer size to a value of 311296 in a disaster-recovery scenario.
If you performed an in-place upgrade of an Exchange 5.5 Server to Exchange 2000 Server, you may notice an unusually high value assigned to the msExchESEParamCacheSizeMax parameter. On some servers, this is a value that is equal to 2,048 MB. In this case, remove or reset the parameter to a more appropriate value. To modify the store database cache size:
- Start the Active Directory Service Interfaces (ADSI) Edit utility. ADSI Edit is included with the Windows Support Tools located in the Support\Tools folder of the Windows 2000 or Windows Server 2003 CD-ROM.
- Under Configuration Container [servername.example.com], where servername.example.com is the fully qualified domain name FQDN of the server, expand CN=Configuration, DC=example, DC=com.
- Expand CN=Services, expand CN=Microsoft Exchange, expand CN=OrganizationName where OrganizationName is the name of your organization, expand CN=Administrative Groups, expand CN=First Administrative Group where First Administrative Group is the name of your administrative group, expand CN=Servers, and then expand CN=servername.
- Under CN=servername, right-click CN=InformationStore, and then click Properties.
- In the Select which properties to view list, click Both.
- In the Select a property to view list, click msExchESEParamCacheSizeMax.
Note The msExchESEParamCacheSizeMax property extends beyond the width of the Select a property to view list. Make sure that you do not unintentionally click the msExchESEParamCacheSizeMin property instead.
- In the Edit Attribute box, type the value that you want to assign this attribute.
Note Make sure that you enter a value that is a multiple of 8,192.
- Click Set, and then click OK.
- Exit the ADSI Edit utility. Then, allow for sufficient time for this value to be replicated throughout the Active Directory forest.
- Restart the Microsoft Exchange Information Store service on the Exchange server.