New functionality in the Distributed Transaction Coordinator service in Windows Server 2003 Service Pack 1 and in Windows XP Service Pack 2

Article translations Article translations
Article ID: 899191 - View products that this article applies to.
Expand all | Collapse all

On This Page

SUMMARY

Microsoft Windows Server 2003 Service Pack 1 (SP1) and Microsoft Windows XP Service Pack 2 (SP2) include many security-related updates and changes. Some of these changes affect the Microsoft Distributed Transaction Coordinator (MSDTC) service.

These changes can be accessed by using the updated Security Configuration dialog box that is available in the Component Services administrative tool.

After you install Windows Server 2003 SP1 or Windows XP SP2, some changes are made to the default security settings that cause Distributed Transaction Coordinator traffic to fail over the network. In this situation, you may receive one or more error messages or error codes.

By modifying the settings in the Security Configuration dialog box, you can help control how the Distributed Transaction Coordinator service communicates with remote computers over the network.

INTRODUCTION

This article describes new functionality in the Microsoft Distributed Transaction Coordinator (MSDTC) service in the following operating systems:
  • Microsoft Windows Server 2003 Service Pack 1 (SP1)
  • Microsoft Windows XP Service Pack 2 (SP2)
The Distributed Transaction Coordinator service coordinates transactions that update two or more transaction-protected resources. Transaction-protected resources include databases, message queues, and file systems. These transaction-protected resources may be located on a single computer or may be distributed between many networked computers.

MORE INFORMATION

In Windows Server 2003 SP1 and in Windows XP SP2, the Distributed Transaction Coordinator service gives you more control over the network communication between computers. By default, all network communication is disabled. The Distributed Transaction Coordinator Security Configuration dialog box has been enhanced so that you can manage these communication settings. To view the Security Configuration dialog box, follow these steps:
  1. Start the Component Services administrative tool. To do this, click Start, click Run, type dcomcnfg.exe, and then click OK.
  2. In the console tree of the Component Services administrative tool, expand Component Services, expand Computers, right-click My Computer, and then click Properties.
  3. Click the MSDTC tab, and then click Security Configuration.

New options that are available in the "Security Configuration" dialog box

The following information describes the new options that are available in the Security Configuration dialog box. This information also describes the registry entries that are affected by the new options in the Security Configuration dialog box.

The "Network DTC Access" check box

The Network DTC Access check box lets you determine whether the Distributed Transaction Coordinator service can access the network. The Network DTC Access check box must be selected together with one of the other check boxes under the Network DTC Access check box to enable network Distributed Transaction Coordinator transactions.

The Network DTC Access check box affects the following registry entry:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC\Security


Value name: NetworkDtcAccess
Value type: REG_DWORD
Value data: 0 (default)
Note On a server cluster, the Network DTC Access check box affects a value in the shared cluster registry key under the MSDTC resource registry key. The registry key of the shared cluster for MSDTC is located at the following location:
HKEY_LOCAL_MACHINE\Cluster\Resources\<MSDTC resource GUID>
By default, the value of the NetworkDtcAccess registry entry is set to 0. A value of 0 turns off the NetworkDtcAccess registry entry. To turn on the NetworkDtcAccess registry entry, set this registry value to 1.

The "Allow Inbound" check box

The Allow Inbound check box lets you determine whether to allow a distributed transaction that originates from a remote computer to run on the local computer. By default, this setting is turned off. To enable this setting, click to select the Network DTC Access check box to set the following registry entry to 1:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC\Security


Value name: NetworkDtcAccess
Value type: REG_DWORD
To disable this setting, click to clear the Network DTC Access check box to set this registry entry to 0.

The Allow Inbound check box affects both of the following registry entries:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC\Security


Value name: NetworkDtcAccessTransactions
Value type: REG_DWORD

Value name: NetworkDtcAccessInbound
Value type: REG_DWORD

The "Allow Outbound" check box

The Allow Outbound check box lets you determine whether to allow the local computer to initiate a transaction and run that transaction on a remote computer. To enable this setting, click to select the Network DTC Access check box to set the following registry entry to 1:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC\Security


Value name: NetworkDtcAccess
Value type: REG_DWORD
To disable this setting, click to clear the Network DTC Access check box to set this registry entry to 0.

The Allow Outbound check box affects both of the following registry entries:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC\Security


Value name: NetworkDtcAccessTransactions
Value type: REG_DWORD

Value name: NetworkDtcAccessOutbound
Value type: REG_DWORD

The "Mutual Authentication Required" option

Mutual Authentication Required adds support for mutual authentication in Windows Server 2003 SP1 and in Windows XP SP2. Mutual Authentication Required sets the greatest security mode that is currently available for network communication. We recommend this transaction mode for client computers that are running Windows XP SP2 together with server computers that are running Windows Server 2003 SP1.

Mutual Authentication Required affects the following registry entries:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC


Value name: AllowOnlySecureRpcCalls
Value type: REG_DWORD
Value data: 1

Value name: FallbackToUnsecureRPCIfNecessary
Value type: REG_DWORD
Value data: 0

Value name: TurnOffRpcSecurity
Value type: REG_DWORD
Value data: 0
Note The functionality that is set by using Mutual Authentication Required differs from the functionality that is set by using Incoming Caller Authentication Required. The three options that are listed under Transaction Manager Communication behave as follows:
  • The Mutual Authentication Required transaction mode requires the remotely accessing component to provide an authenticated connection with the local computer. This authentication is verified by impersonation on the local computer. Additionally, if the remote access communication is performed between two Distributed Transaction Coordinator services, this authentication information must specify a computer account that matches the remote transaction mode computer's host name.
  • The Incoming Caller Authentication Required transaction mode only requires the remote connection to be authenticated. Additionally, if the remotely accessing component is a Distributed Transaction Coordinator service, the authentication information must be for a computer account.
  • The No Authentication Required transaction mode does not validate an authenticated connection or verify whether an authenticated connection is being established.
In a clustered environment, the computer account for the Distributed Transaction Coordinator service specifies the cluster node's host name. In a clustered environment, the Distributed Transaction Coordinator authentication does not use the transaction mode's host name. In a clustered environment, the transaction mode's host name is the name of the virtual service. Therefore, you cannot use the Mutual Authentication Required transaction mode in a clustered environment, or on any computers that are negotiating transactions with such computers. You can use the  Mutual Authentication Required transaction mode between two nonclustered computers that are running Windows Server 2003 SP1 or between two computers that are running Windows XP SP2.

You must use the Incoming Caller Authentication Required transaction mode between Windows Server 2003-based computers in a clustered environment.

You must use the No Authentication Required transaction mode where one or more of the following conditions are true:
  • The network access is between computers that are running Microsoft Windows 2000.
  • The network access is between two domains that do not have a mutual trust configured.
  • The network access is between computers that are members of a workgroup.

The "Incoming Caller Authentication Required" option

Incoming Caller Authentication Required requires the local Distributed Transaction Coordinator service to communicate with a remote Distributed Transaction Coordinator service by using only encrypted messages. Only the incoming connection will be authenticated. Only Windows Server 2003 SP1 and Windows XP SP2 support this feature. Therefore, only enable this option if the remote Distributed Transaction Coordinator service is running on a Windows Server 2003 SP1-based computer or on a Windows XP SP2-based computer.

Incoming Caller Authentication Required affects the following registry entries:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC


Value name: AllowOnlySecureRpcCalls
Value type: REG_DWORD
Value data: 0

Value name: FallbackToUnsecureRPCIfNecessary
Value type: REG_DWORD
Value data: 1

Value name: TurnOffRpcSecurity
Value type: REG_DWORD
Value data: 0
For more information about Incoming Caller Authentication Required, see the "The Mutual Authentication Required option" section.

The "No Authentication Required" option

No Authentication Required enables operating system compatibility between earlier versions of the Windows operating system. When this option is enabled, network communication between Distributed Transaction Coordinator services can fall back to nonauthenticated communication or to nonencrypted communication if a secure communication channel cannot be established.

Note We recommend that you use this setting if the remote Distributed Transaction Coordinator service is running on a computer that is running Microsoft Windows 2000 or on a computer that is running a version of Windows XP that is earlier than Windows XP SP2.

You can also use No Authentication Required to resolve a situation where the Distributed Transaction Coordinator services are running on computers that are in domains that do not have a trust relationship established. Additionally, you can use No Authentication Required to resolve a situation where the Distributed Transaction Coordinator services are running on computers that are members of a workgroup.

No Authentication Required affects the following registry entries:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC


Value name: AllowOnlySecureRpcCalls
Value type: REG_DWORD
Value data: 0

Value name: FallbackToUnsecureRPCIfNecessary
Value type: REG_DWORD
Value data: 0

Value name: TurnOffRpcSecurity
Value type: REG_DWORD
Value data: 1
Note On a server cluster, these registry entries are located in the shared cluster registry.

Significance of the new options that are available in the "Security Configuration" dialog box

The new options that are available in the Security Configuration dialog box let you apply security settings to outgoing or incoming network communications. By default, after you install Windows Server 2003 SP1 or Windows XP SP2, the computer does not accept network traffic. Therefore, the computer is less vulnerable to network access by a malicious user. Additionally, the protocols that are sent over the network are updated to support a more securely encrypted and mutually authenticated communications mode. This helps reduce the chance that a malicious user could intercept and take over communications between Distributed Transaction Coordinator services.

Network communication changes in Windows Server 2003 SP1 and in Windows XP SP2

After you install Windows Server 2003 SP1 or Windows XP SP2, all network communication coming out of the Distributed Transaction Coordinator service or coming in to the Distributed Transaction Coordinator service is disabled. For example, if a COM+ object tries to update a Microsoft SQL Server database that is located on a remote computer by using a Distributed Transaction Coordinator transaction, this transaction does not succeed. Conversely, if the computer hosts a SQL Server database that components from a remote computer try to access by using a Distributed Transaction Coordinator transaction, this transaction does not succeed.

Issues that are related to the Distributed Transaction Coordinator service

Transactions fail because of network connectivity issues

Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:
322756 How to back up and restore the registry in Windows


If the Distributed Transaction Coordinator transactions fail because of network connectivity issues, click to select the following check boxes in the Security Configuration dialog box:
  • Click to select the Network DTC Access check box.
  • Click to select one or both of the following check boxes under Transaction Manager Communication depending on your requirements:
    • Allow Inbound
    • Allow Outbound
If you want to programmatically change these settings as part of a Windows Server 2003 SP1 or Windows XP SP2 deployment, you can directly modify the registry settings that correspond to the settings that you want to set. After you modify the registry settings, you must restart the Distributed Transaction Coordinator service.

Important We recommend that you do not manually modify the registry to change these settings. If you manually modify these registry settings, you may experience issues with the Cluster service on Windows Server 2003 SP1-based server clusters.

Windows Firewall blocks Distributed Transaction Coordinator traffic

Important These steps may increase your security risk. These steps may also make the computer or the network more vulnerable to attack by malicious users or by malicious software such as viruses. We recommend the process that this article describes to enable programs to operate as they are designed to or to implement specific program capabilities. Before you make these changes, we recommend that you evaluate the risks that are associated with implementing this process in your particular environment. If you decide to implement this process, take any appropriate additional steps to help protect the system. We recommend that you use this process only if you really require this process.

If you use Windows Firewall to help protect Windows Server 2003 SP1 or Windows XP SP2, you must add the Distributed Transaction Coordinator service to the exception list in the Windows Firewall settings. To do this, follow these steps:
  1. Click Start, click Run, type firewall.cpl, and then click OK.
  2. In the Windows Firewall dialog box, click the Exceptions tab, and then click Add Program.
  3. Click Browse, locate and then click C:\Windows\System32\msdtc.exe, and then click Open.
  4. Click OK, click to select the msdtc.exe check box in the Programs and Services list if this check box is not already selected, and then click OK.

Settings that are changed or added in Windows Server 2003 SP1 or in Windows XP SP2

The following table describes the registry entries that are changed in Windows XP SP2 from earlier versions of Windows.
Collapse this tableExpand this table
Entry nameLocationPrevious default valueWindows XP SP2 default valuePossible values
NetworkDtcAccess HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC\Security 100 or 1
NetworkDtcAccessTransactions HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC\Security 100 or 1
NetworkDtcAccessInboundHKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC\Security Not applicable00 or 1
NetworkDtcAccessOutboundHKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC\Security Not applicable00 or 1
AllowOnlySecureRpcCallsHKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC Not applicable10 or 1
FallbackToUnsecureRPCIfNecessaryHKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC Not applicable00 or 1
TurnOffRpcSecurityHKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC Not applicable00 or 1
Note These changes appear in the shared cluster registry on a Windows Server 2003 SP1-based server cluster.

Error codes that are associated with the Distributed Transaction Coordinator service changes in Windows XP SP2

After you install Windows XP SP2, you may receive one of the following error codes when you run Distributed Transaction Coordinator transactions between computers:

Error code 1
//

// MessageId: XACT_E_NETWORK_TX_DISABLED

//

// MessageText:

//

// The transaction manager has disabled its support for remote/network transactions.

//

#define XACT_E_NETWORK_TX_DISABLED       _HRESULT_TYPEDEF_(0x8004D024L)

Error code 2
//

// MessageId: XACT_E_PARTNER_NETWORK_TX_DISABLED

//

// MessageText:

//

// The partner transaction manager has disabled its support for remote/network transactions.

//

#define XACT_E_PARTNER_NETWORK_TX_DISABLED _HRESULT_TYPEDEF_(0x8004D025L)

Properties

Article ID: 899191 - Last Review: February 15, 2011 - Revision: 3.0
APPLIES TO
  • Microsoft Windows Server 2003 Service Pack 1, when used with:
    • Microsoft Windows Server 2003, Enterprise Edition (32-bit x86)
    • Microsoft Windows Server 2003, Standard Edition (32-bit x86)
    • Microsoft Windows Server 2003, Web Edition
Keywords: 
kbinfo KB899191

Give Feedback

 

Contact us for more help

Contact us for more help
Connect with Answer Desk for expert help.
Get more support from smallbusiness.support.microsoft.com