How to optimize memory usage in Exchange Server 2003

Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:

This article describes the optimization of memory usage on your computer that is running Exchange Server 2003.

If you have 1 gigabyte (GB) or more of physical memory (RAM) installed on a server that is running Microsoft Exchange Server 2003, you must make sure that Exchange 2003 can make efficient use of that memory.

Note Exchange 2003 performs an optimal memory configuration check when the store process starts. If the memory settings are not optimal, you will see event 9665 in Event Viewer. This message appears if one or more of the following conditions are true:

• The server is running any edition of Microsoft Windows 2000 Server and the SystemPages value in the registry is set outside the range of 24000 to 31000.
  
  Notes
  • The SystemPages subkey is in the following location: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\SystemPages
• The server is running Microsoft Windows 2000 Advanced Server or Microsoft Windows 2000 Datacenter Server, has 1 GB or more of physical memory, but does not have the /3GB switch set in the boot.ini file.
• The server is running Microsoft Windows Server 2003 Standard, Microsoft Windows Server 2003 Enterprise or Microsoft Windows Server 2003 Datacenter edition, and the SystemPages value in the registry is set to something other than 0.
• The server is running Microsoft Windows Server 2003 Standard Edition, Microsoft Windows Server 2003 Enterprise Edition or Microsoft Windows Server 2003 Datacenter Edition, has 1 GB or more of physical memory and the /3GB switch is set, but the /USERVA setting is not present in the boot.ini file.
• The server is running any edition of Microsoft Windows 2000 Server or Microsoft Windows Server 2003, and the HeapDeCommitFreeBlockThreshold value in the registry is set to something other than 0x00040000.
  
  Note 0x00040000 is a hex value. You should type 40000 (HEX) in the registry for the HeapDeCommitFreeBlockThreshold value.

The memory configuration check does not occur on servers that are running Microsoft Small Business Server. When the 9665 event is raised, a DWORD is written to the event log as additional data.

If you want to turn off the memory configuration check, you can create the following registry key: Parameter: Suppress Memory Configuration Notification
Type: REG_DWORD
Setting: 1

Notes

• For Exchange Server computers that do not contain any mailboxes or public folders, such as mail gateways, we do not recommend setting the /3GB switch in boot.ini, independently of installed physical memory size.
• For Exchange Server computers which are at the same time Active Directory Domain Controllers or Global Catalog servers we also do not recommend setting the /3GB switch in boot.ini. We recommend having dedicated Active Directory Domain Controllers or Global Catalog servers.
• Because Microsoft Small Business Server installs as an integrated domain controller and Exchange server, we also do not recommend setting the /3GB switch when you run Exchange Server 2003 in a Small Business Server environment. However, the other concepts and settings that are described in this article apply equally to Exchange Server 2003 in a Microsoft Small Business Server environment.

The following sections in this article contain recommendations for each of these settings.

Virtual address space concepts

Independent of the hardware configuration, the number of databases, and the number of users on the server, the Exchange Information Store (Store.exe) process in Exchange 2003 has a finite amount of memory that it can address. This amount is known as the virtual address space. In most scenarios, the usage of this virtual address space for the information store dictates the overall performance and scalability of the Exchange Server 2003 mailbox servers. For small to medium-sized servers, Exchange Server 2003 automatically makes the best balance. However, for larger servers you may want to manually adjust some of the tuning parameters.

If the Exchange Server 2003 computer has 1 gigabyte (GB) of memory or more installed and if the computer is home to mailboxes or public folders, make sure that you add the /3GB switch to the Boot.ini file on the server. If the server does not have any mailboxes or public folders on it, such as a mail gateway, we do not recommend that you use the /3GB switch. By default, the Microsoft Windows 2000 Advanced Server and Windows Server 2003 operating systems reserve 2 GB of virtual address space for Kernel mode usage, and 2 GB for User mode. Virtual address space for a specific process is allocated at startup and increases as more memory is used during operation. Typically, the actual memory usage (the working set) of a process is much less than the address space allocated to that process. On a computer that is running Exchange Server 2003 with 1 GB or more of memory, you must modify the Windows 2000 Advanced Server and Windows Server 2003 operating systems so that 3 GB of User mode memory space is available. You can do this by using the /3GB switch in the Boot.ini file. To use this setting on a Windows 2000 Advanced Server-based computer, you must also configure the SystemPages registry in the following registry subkey:Set the SystemPages registry entry to a DWORD value between 24000 and 31000 (decimal), and then click OK. If you are unsure which value to use, we recommend that you use 31000 (decimal). For more information, visit the following Microsoft Web site:This registry change is not required, nor do we recommend it on Windows 2003-based servers because this functionality is achieved by using the Userva switch that is outlined in a later section.

Do not set the /3GB switch if you are running Windows 2000 Server Standard Edition. This memory tuning switch is not supported on Windows 2000 Server Standard Edition. Although the server will not generate an error message if you do this, the result of setting this switch is that a false memory address space will exist. In cases where a process tries to access this higher address space, a Stop error message on a blue screen occurs, and the server stops responding.

Note The /3GB tuning switch is supported on all releases of Windows Server 2003 including Windows Server 2003 Standard Edition. For more information about how to set the /3GB switch, click the following article number to view the article in the Microsoft Knowledge Base: If you are running Exchange 2003 on a Windows Server 2003-based computer, and if the /3GB switch is set, we recommend that you set the /USERVA=3030 parameter in the Boot.ini file. This enables more system page table entries (PTE) on the server. For more information about the /USERVA switch, click the following article number to view the article in the Microsoft Knowledge Base: Make sure that the Store.exe process does not run out of virtual address space. When the virtual address space is exhausted, memory allocations will be unsuccessful even if there is a lot of physical RAM available. To resolve this issue, you must restart the Information Store service. For example, a server that has 2 GB of physical RAM without the /3GB switch set in the Boot.ini file will run out of memory when the virtual address space for the Store.exe process reaches 2 GB. In this case, Windows Task Manager may show that only about 1.5 GB of memory is actually being used. However, the server is out of memory and the Information Store service must be restarted.

When virtual memory in the Store.exe process runs low, the performance of the Exchange Server 2003 server can decrease dramatically. When the largest free block of virtual memory is reduced to 32 MB, an Event ID 9582 Warning event is generated in the Application log of the Event Viewer. When you see this event, it is best to restart the Store.exe process at the next available opportunity. If the largest block of memory is reduced even more, to the 16 MB level, an Event ID 9582 Error event is generated in the Application log of the Event Viewer. If this event occurs, the server is approaching a critical operating condition and must be restarted at the next opportunity. After this error occurs, memory may become exhausted in several hours. If you do not respond to these events, you may experience one or more of the following symptoms:

• Intermittent message delivery failures.
• Intermittent failure of the Internet mail (IMAIL) conversion process that is indicated by Event ID 12800.
• Certain Exchange-aware antivirus programs may generate error messages or events.

Maximizing virtual address space

It is best to monitor the virtual address space for the Information Store on large mailbox servers to make sure that performance and stability does not decrease. The easiest way to do this is to monitor the VM Largest Block Size counter of the MSExchangeIS performance object in the Performance utility. This value is shown in bytes. After you start the computer, it is typical to see a sharp decrease in the largest virtual memory block size. However, after a day or two of operation, the value approaches a typical operating level. A value for the largest free virtual memory block that is more than 200,000,000 bytes (about 200 MB) indicates a healthy server. If you notice a lower value than this, monitor the server more closely. If you experience a low virtual address space:

1. If you have 1 GB or more of physical memory installed, make sure that the /3GB switch is set in the Boot.ini file if the following conditions are true on the server:
   • The server is home to mailboxes or public folders.
   • The operating system is Windows 2000 Advanced Server or Windows Server 2003.
   If the server does not have any mailboxes or public folders on it, such as a mail gateway, we do not recommend that you use the /3GB switch.
2. If Exchange Server 2003 is installed on a Windows Server 2003-based server, and if it has the /3GB switch set, make sure that the /USERVA=3030 switch is set in the Boot.ini file. If Exchange Server 2003 is installed on a Windows 2000-based server, make sure that the server is running Windows 2000 Service Pack 3 (SP3) or later.
3. If you have 1 GB or more physical memory installed, configure the HeapDeCommitFreeBlockThreshold registry value in the following registry key, and then restart the server:
   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager
   Value name: HeapDeCommitFreeBlockThreshold
   Value type: REG_DWORD
   Value data: 0x00040000 (recommended)
   Value default: not present
   For more information about how to use and configure this registry value, click the following article number to view the article in the Microsoft Knowledge Base:
   315407  (http://support.microsoft.com/kb/315407/ ) The "HeapDecommitFreeBlockThreshold" registry key

Typically, the previous steps resolve the virtual address space consumption issue. However, on some larger servers, you may have to perform additional tuning of the Store Database Cache Size to re-align the balance between performance and scalability.

Store Database Cache size

The Store Database Cache is also known as the ESE buffer, and it provides a large caching area for database transactions before they are committed to the store. By default, Exchange Server 2003 queries the memory configuration of the local computer, and then allocates 896 MB of RAM if the /3GB switch is set in the Boot.ini file, or 576 MB of RAM if the /3GB switch is not set. Where a server is heavily loaded or where disk performance is not optimal, a large ESE buffer increases overall system performance. Depending on your configuration, you may have to increase or reduce the size of this buffer to obtain the best overall performance.

In a scenario where Exchange Server 2003 is used in an environment where it co-exists with other server-side programs, it may monopolize the available memory resources. The Dynamic Buffer Allocation (DBA) algorithm is responsible for returning memory to the operating system if other programs require it. However, you can manually limit the memory that Exchange Server 2003 uses by reducing the ESE buffer.

Warning On servers that have more than 2 GB of memory, it may help increase the size of the ESE buffer. Because of virtual address space limitations, this value must not be set greater than 1,200 MB.

Before you increase the maximum buffer size, we recommend that you use the Windows Performance utility to monitor the memory of the server under a typical load. To do this, monitor the following performance object and value:The information gathered from the performance monitoring gives you an accurate value for the virtual address space that the Store.exe process has allocated. On a server with the /3GB switch set in the Boot.ini file, the value that is seen in the Performance utility is typically less than 2.5 GB. On a server without the /3GB switch set in the Boot.ini file, the value is typically less than 1.8 GB. It is best to add the /3GB switch to the Boot.ini file on servers with 1 GB or more of installed memory. If you see values that are larger than those previously noted, for either configuration, do not increase the size of your maximum buffer size. If you see values that are less than previously noted, for either configuration, you may want to increase the size of the database maximum buffer size.

For example, if you have a server that is configured to use the /3GB switch in the Boot.ini file, and performance monitoring shows the virtual bytes count at 2.5 GB when the server is heavily loaded, you may be able to increase your maximum buffer size by about 300 MB, for a total size of 1,200 MB.

Increasing the buffer size may adversely affect server performance. A larger buffer means that more virtual address space is consumed. Therefore, if the server experiences virtual memory address space constraints, increasing the buffer size may create operating system instability. This may lead to an unresponsive server. On a very large mailbox server, you may have to decrease the default buffer size to prevent system instability.

How to modify the ESE buffer size

The msExchESEParamCacheSizeMax parameter controls the ESE buffer size. Its value is expressed as a page count, and must be set to an exact multiple of 8192 for maximum efficiency:

• The default size on servers with the /3GB switch set: 229376 (896 MB)
• The default size on servers without the /3GB switch set: 147456 (576 MB)
• The recommended maximum value with the /3GB switch set: 311296 (1.2 GB)
  
  Note You should first monitor the virtual bytes for the store process as stated earlier in this article before increasing this value over the default.
• The recommended maximum value without the /3GB switch set: 196608 (768 MB)
  
  Note You should first monitor the virtual bytes for the store process as stated earlier in this article before increasing this value over the default.

Note The replay of transaction logs is significantly faster when the ESE buffer is set to a large size. You may want to temporarily increase the ESE buffer size to a value of 311296 in a disaster-recovery scenario.

If you performed an in-place upgrade of an Exchange 5.5 Server computer to Exchange 2000 Server, you may notice an unusually high value assigned to the msExchESEParamCacheSizeMax parameter. On some servers, this is a value that is equal to 2,048 MB. In this case, remove or reset the parameter to a more appropriate value. To modify the store database cache size:

1. Start the Active Directory Service Interfaces (ADSI) Edit utility. ADSI Edit is included with the Windows Support Tools located in the Support\Tools folder of the Windows 2000 or Windows Server 2003 CD-ROM.
2. Under Configuration Container [servername.example.com], where servername.example.com is the fully qualified domain name FQDN of the server, expand CN=Configuration, DC=example, DC=com.
3. Expand CN=Services, expand CN=Microsoft Exchange, expand CN=OrganizationName where OrganizationName is the name of your organization, expand CN=Administrative Groups, expand CN=First Administrative Group where First Administrative Group is the name of your administrative group, expand CN=Servers, and then expand CN=servername.
4. Under CN=servername, right-click CN=InformationStore, and then click Properties.
5. In the Select which properties to view list, click Both.
6. In the Select a property to view list, click msExchESEParamCacheSizeMax.
   
   Note The msExchESEParamCacheSizeMax property extends beyond the width of the Select a property to view list. Make sure that you do not unintentionally click the msExchESEParamCacheSizeMin property instead.
7. In the Edit Attribute box, type the value that you want to assign this attribute.
   
   Note Make sure that you enter a value that is a multiple of 8,192.
8. Click Set, and then click OK.
9. Quit the ADSI Edit utility. Then, allow for sufficient time for this value to be replicated throughout the Active Directory forest.
10. Restart the Microsoft Exchange Information Store service on the Exchange server.

Removing unnecessary video drivers

An Exchange server will not typically require a powerful graphics card or driver. You should make sure that the only the default VGA driver is installed on the server. To do this, remove any third-party drivers, and then verify that only the default VGA driver is installed. You should add the /BASEVIDEO switch to the Boot.ini startup parameters. This will also force windows to use a less resource-intensive default VGA driver. This has been found to save around 1000 System PTEs or more.