New functionality in the Distributed Transaction Coordinator service in Windows Server 2003 Service Pack 1 and in Windows XP Service Pack 2

This article describes new functionality in the Microsoft Distributed Transaction Coordinator (MSDTC) service in the following operating systems:

• Microsoft Windows Server 2003 Service Pack 1 (SP1)
• Microsoft Windows XP Service Pack 2 (SP2)

The Distributed Transaction Coordinator service coordinates transactions that update two or more transaction-protected resources. Transaction-protected resources include databases, message queues, and file systems. These transaction-protected resources may be located on a single computer or may be distributed between many networked computers.

In Windows Server 2003 SP1 and in Windows XP SP2, the Distributed Transaction Coordinator service gives you more control over the network communication between computers. By default, all network communication is disabled. The Distributed Transaction Coordinator Security Configuration dialog box has been enhanced so that you can manage these communication settings. To view the Security Configuration dialog box, follow these steps:

1. Start the Component Services administrative tool. To do this, click Start, click Run, type dcomcnfg.exe, and then click OK.
2. In the console tree of the Component Services administrative tool, expand Component Services, expand Computers, right-click My Computer, and then click Properties.
3. Click the MSDTC tab, and then click Security Configuration.

New options that are available in the "Security Configuration" dialog box

The following information describes the new options that are available in the Security Configuration dialog box. This information also describes the registry entries that are affected by the new options in the Security Configuration dialog box.

The "Network DTC Access" check box

The Network DTC Access check box lets you determine whether the Distributed Transaction Coordinator service can access the network. The Network DTC Access check box must be selected together with one of the other check boxes under the Network DTC Access check box to enable network Distributed Transaction Coordinator transactions.

The Network DTC Access check box affects the following registry entry: Note On a server cluster, the Network DTC Access check box affects a value in the shared cluster registry key under the MSDTC resource registry key. The registry key of the shared cluster for MSDTC is located at the following location:By default, the value of the NetworkDtcAccess registry entry is set to 0. A value of 0 turns off the NetworkDtcAccess registry entry. To turn on the NetworkDtcAccess registry entry, set this registry value to 1.

The "Allow Inbound" check box

The Allow Inbound check box lets you determine whether to allow a distributed transaction that originates from a remote computer to run on the local computer. By default, this setting is turned off. To enable this setting, click to select the Network DTC Access check box to set the following registry entry to 1: To disable this setting, click to clear the Network DTC Access check box to set this registry entry to 0.

The Allow Inbound check box affects both of the following registry entries:

The "Allow Outbound" check box

The Allow Outbound check box lets you determine whether to allow the local computer to initiate a transaction and run that transaction on a remote computer. To enable this setting, click to select the Network DTC Access check box to set the following registry entry to 1: To disable this setting, click to clear the Network DTC Access check box to set this registry entry to 0.

The Allow Outbound check box affects both of the following registry entries:

The "Mutual Authentication Required" option

Mutual Authentication Required adds support for mutual authentication in Windows Server 2003 SP1 and in Windows XP SP2. Mutual Authentication Required sets the greatest security mode that is currently available for network communication. We recommend this transaction mode for client computers that are running Windows XP SP2 together with server computers that are running Windows Server 2003 SP1.

Mutual Authentication Required affects the following registry entries: Note The functionality that is set by using Mutual Authentication Required differs from the functionality that is set by using Incoming Caller Authentication Required. The three options that are listed under Transaction Manager Communication behave as follows:

• The Mutual Authentication Required transaction mode requires the remotely accessing component to provide an authenticated connection with the local computer. This authentication is verified by impersonation on the local computer. Additionally, if the remote access communication is performed between two Distributed Transaction Coordinator services, this authentication information must specify a computer account that matches the remote transaction mode computer's host name.
• The Incoming Caller Authentication Required transaction mode only requires the remote connection to be authenticated. Additionally, if the remotely accessing component is a Distributed Transaction Coordinator service, the authentication information must be for a computer account.
• The No Authentication Required transaction mode does not validate an authenticated connection or verify whether an authenticated connection is being established.

In a clustered environment, the computer account for the Distributed Transaction Coordinator service specifies the cluster node's host name. In a clustered environment, the Distributed Transaction Coordinator authentication does not use the transaction mode's host name. In a clustered environment, the transaction mode's host name is the name of the virtual service. Therefore, you cannot use the Mutual Authentication Required transaction mode in a clustered environment. You can use the Mutual Authentication Required transaction mode between two nonclustered computers that are running Windows Server 2003 SP1 or between two computers that are running Windows XP SP2.

You must use the Incoming Caller Authentication Required transaction mode between Windows Server 2003-based computers in a clustered environment.

You must use the No Authentication Required transaction mode where one or more of the following conditions are true:

• The network access is between computers that are running Microsoft Windows 2000.
• The network access is between two domains that do not have a mutual trust configured.
• The network access is between computers that are members of a workgroup.

The "Incoming Caller Authentication Required" option

Incoming Caller Authentication Required requires the local Distributed Transaction Coordinator service to communicate with a remote Distributed Transaction Coordinator service by using only encrypted messages. Only the incoming connection will be authenticated. Only Windows Server 2003 SP1 and Windows XP SP2 support this feature. Therefore, only enable this option if the remote Distributed Transaction Coordinator service is running on a Windows Server 2003 SP1-based computer or on a Windows XP SP2-based computer.

Incoming Caller Authentication Required affects the following registry entries: For more information about Incoming Caller Authentication Required, see the "The Mutual Authentication Required option" section.

The "No Authentication Required" option

No Authentication Required enables operating system compatibility between earlier versions of the Windows operating system. When this option is enabled, network communication between Distributed Transaction Coordinator services can fall back to nonauthenticated communication or to nonencrypted communication if a secure communication channel cannot be established.

Note We recommend that you use this setting if the remote Distributed Transaction Coordinator service is running on a computer that is running Microsoft Windows 2000 or on a computer that is running a version of Windows XP that is earlier than Windows XP SP2.

You can also use No Authentication Required to resolve a situation where the Distributed Transaction Coordinator services are running on computers that are in domains that do not have a trust relationship established. Additionally, you can use No Authentication Required to resolve a situation where the Distributed Transaction Coordinator services are running on computers that are members of a workgroup.

No Authentication Required affects the following registry entries: Note On a server cluster, these registry entries are located in the shared cluster registry.

Significance of the new options that are available in the "Security Configuration" dialog box

The new options that are available in the Security Configuration dialog box let you apply security settings to outgoing or incoming network communications. By default, after you install Windows Server 2003 SP1 or Windows XP SP2, the computer does not accept network traffic. Therefore, the computer is less vulnerable to network access by a malicious user. Additionally, the protocols that are sent over the network are updated to support a more securely encrypted and mutually authenticated communications mode. This helps reduce the chance that a malicious user could intercept and take over communications between Distributed Transaction Coordinator services.

Network communication changes in Windows Server 2003 SP1 and in Windows XP SP2

After you install Windows Server 2003 SP1 or Windows XP SP2, all network communication coming out of the Distributed Transaction Coordinator service or coming in to the Distributed Transaction Coordinator service is disabled. For example, if a COM+ object tries to update a Microsoft SQL Server database that is located on a remote computer by using a Distributed Transaction Coordinator transaction, this transaction does not succeed. Conversely, if the computer hosts a SQL Server database that components from a remote computer try to access by using a Distributed Transaction Coordinator transaction, this transaction does not succeed.

Issues that are related to the Distributed Transaction Coordinator service

Transactions fail because of network connectivity issues

Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:

If the Distributed Transaction Coordinator transactions fail because of network connectivity issues, click to select the following check boxes in the Security Configuration dialog box:

• Click to select the Network DTC Access check box.
• Click to select one or both of the following check boxes under Transaction Manager Communication depending on your requirements:
  • Allow Inbound
  • Allow Outbound

If you want to programmatically change these settings as part of a Windows Server 2003 SP1 or Windows XP SP2 deployment, you can directly modify the registry settings that correspond to the settings that you want to set. After you modify the registry settings, you must restart the Distributed Transaction Coordinator service.

Important We recommend that you do not manually modify the registry to change these settings. If you manually modify these registry settings, you may experience issues with the Cluster service on Windows Server 2003 SP1-based server clusters.

Windows Firewall blocks Distributed Transaction Coordinator traffic

Important These steps may increase your security risk. These steps may also make the computer or the network more vulnerable to attack by malicious users or by malicious software such as viruses. We recommend the process that this article describes to enable programs to operate as they are designed to or to implement specific program capabilities. Before you make these changes, we recommend that you evaluate the risks that are associated with implementing this process in your particular environment. If you decide to implement this process, take any appropriate additional steps to help protect the system. We recommend that you use this process only if you really require this process.

If you use Windows Firewall to help protect Windows Server 2003 SP1 or Windows XP SP2, you must add the Distributed Transaction Coordinator service to the exception list in the Windows Firewall settings. To do this, follow these steps:

1. Click Start, click Run, type firewall.cpl, and then click OK.
2. In the Windows Firewall dialog box, click the Exceptions tab, and then click Add Program.
3. Click Browse, locate and then click C:\Windows\System32\msdtc.exe, and then click Open.
4. Click OK, click to select the msdtc.exe check box in the Programs and Services list if this check box is not already selected, and then click OK.

Settings that are changed or added in Windows Server 2003 SP1 or in Windows XP SP2

The following table describes the registry entries that are changed in Windows XP SP2 from earlier versions of Windows. Note These changes appear in the shared cluster registry on a Windows Server 2003 SP1-based server cluster.

Error codes that are associated with the Distributed Transaction Coordinator service changes in Windows XP SP2

After you install Windows XP SP2, you may receive one of the following error codes when you run Distributed Transaction Coordinator transactions between computers:

Error code 1Error code 2