Removing duplicate and unwanted proxy addresses in Exchange

If you are an administrator, you can use Microsoft Exchange recipient policies to automatically configure and customize e-mail addresses for all Exchange recipients by using address generation rules. The Recipient Update Service applies addresses in bulk to new and existing users according to the specified rules. To configure these rules, use Exchange System Manager to access the properties of recipient policy objects.

However, in some situations, you may use a rule that causes duplicate or that causes multiple addresses to be applied across the Exchange organization. When duplicate e-mail addresses are detected during an ordinary Exchange operation, non-delivery reports (NDRs) with a 5.1.4 error code may be sent by the server, and you may experience other unexpected behavior. Additionally, the following event ID message may be logged in the Application Event log:

Event Type: Warning
Event Source: MSExchangeIS
Event Category: General
Event ID: 9514
Computer: Exchange_Server_Name
Description: Two objects in the Directory have the same proxy - /dc=com/dc=domain/cn=configuration/cn=services/cn=microsoft exchange/cn=organization_name/cn=administrative groups/cn=administrative_group_name/cn=servers/cn=Exchange_server_name/cn=informationstore/cn=storage_group_name/cn=public folder store (Exchange_server_name) and /dc=com/dc=domain/ou=users/cn=user_name.

This article describes a situation where duplicate addresses are applied and how to remove these duplicate addresses.

The following rule is a typical rule for an SMTP (Internet) mail format address: When you use this rule, an e-mail address is added to each recipient object that is covered by the recipient policy of mailnickname@domain.com. Therefore, if a user's Exchange e-mail alias is user1, the address user1@domain.com is applied to the user.

You can also use Exchange to define auto-naming rules. For example, you may want the Internet e-mail address for recipients to be configured as "FirstName_LastName@domain.com" instead of "mailnickname@domain.com." You can use the following address generation rule to make this change: For additional information about available auto-naming specifiers and syntax, click the following article number to view the article in the Microsoft Knowledge Base: If you are using auto-naming specifiers, you must make sure that the fields that you reference in the address generation rule actually exist for each of the recipients that are covered by the rule. If you do not do so, you may cause duplicate or multiple e-mail addresses to be generated on objects throughout the Exchange organization.

The effects of this problem vary depending on what recipients are involved. Several Exchange system objects are mailbox-enabled. If the wrong address is applied to these system objects, problems may occur. For example, public folder replication may be unsuccessful or databases may not be able to mount. Additionally, the event ID message that is described in the "Summary" section of this article may be logged.

As an example of this behavior, suppose that you have defined the %g_%s@domain.com address generation rule. This rule takes the values for the givenName attribute and the sn attribute of an Active Directory object and uses these values to construct an e-mail address. If a user's givenName attribute value is Jeff and the sn attribute value is Smith, the e-mail address is Jeff_Smith@domain.com.

But the givenName attribute and the sn attribute are not mandatory, and mail-enabled objects such as distribution lists and system agents may lack these attributes. In this situation, the %g_%s@domain.com address generation rule devolves to the _@domain.com address generation rule. For objects that do not have the givenName and the sn attribute value, the rule becomes the equivalent of a rule to hardcode the e-mail address as _@domain.com.

When you apply new e-mail addresses, the Recipient Update Service checks to see if the same address already exists on any object in the forest. If it does, a numeric value is added to the new address to differentiate it. For example, if there are multiple users named Jeff Smith, an additional user account that is processed by the Recipient Update Service may receive an e-mail address of Jeff_Smith2@domain.com.

This means that objects without the givenName and the sn attribute values who are processed according to the %g_%s@domain.com address generation rule will receive e-mail addresses such as _1@domain.com, _2@domain.com, _3@domain.com, and other similar addresses.

Every time that the Recipient Update Service checks for new recipients who must have e-mail addresses, objects without the givenName and the sn attributes may receive an additional e-mail address. Consider an object that has the _1@domain.com address. This address does not match the _@domain.com rule, and the Recipient Update Service may assume that the object still must have the "hardcoded" _@domain.com address applied. Duplicate address checking finds that the_@domain.com address already belongs to a different object, and therefore, an address such as _4@domain.com may be assigned.

If none of the attributes of an object have been modified in Active Directory since the last time the Recipient Update Service ran, that object is not noticed by the Recipient Update Service. But if the object is changed in any way, the Recipient Update Service scans it again to determine if new e-mail addresses will be applied to it. This means that over time, a single object that lacks the specifier attributes can have dozens or hundreds of _NNNN@domain.com e-mail addresses.

Generally, the additional addresses are inert and do not affect ordinary mail flow; however, over time they can cause the Recipient Update Service to process items more and more inefficiently. The Recipient Update Service must check for duplicate addresses repeatedly when it assigns new addresses to objects. If the _1000@domain.com address exists, the Recipient Update Service must perform over a thousand duplicate address checks before it can assign a new address based on the _@domain.com generation rule.

Although Exchange checks for duplicate e-mail addresses, the check is not foolproof for the following two reasons:

• A separate Recipient Update Service must be configured for each domain. Even in a single domain environment, there are two Recipient Update Services. One Recipient Update Service is for the domain container and the other (the enterprise Recipient Update Service) is for the server Configuration container.
• Active Directory replication latency may cause an address that is assigned by one Recipient Update Service not to appear in the folder database that is being used by a different Recipient Update Service at the appropriate time.

The most likely scenario where duplicate addresses are created is a scenario when the enterprise Recipient Update Service is connected to one domain controller and the domain Recipient Update Service is connected to another. When objects that are lacking the specifier attributes are processed in parallel by each Recipient Update Service, duplicates are generated.

Best Practices to Prevent Duplicate and Multiple Addresses

The character that is used most frequently in auto-naming rules is underscore character (_). This section refers to the address generation rule SMTP:%g_%s@domain.com as the base example.

A %g%s@domain.com address generation rule does not cause the same problems as %g_%s@domain.com. If an object lacks the givenName and the sn attributes, the address generation rule devolves to @domain.com. This invokes the default Recipient Update Service behavior of stamping an e-mail address of mailNickname@domain.com

To prevent auto-naming specifiers from causing these problems if you are using auto-naming specifiers and you are using hard characters in the user part of the address generation rule, you must configure the filter on the recipient policy to explicitly test for the existence of the attributes that you specify.
Each Exchange recipient policy has an Lightweight Directory Access Protocol (LDAP) filter that defines the objects that the policy applies to. The simplest filter you can define on an object is (mailnickname=*). In LDAP filter syntax, attribute=* can be read as "attribute exists." Therefore, the (mailnickname=*) filter applies the policy to every object that has a mailNickname attribute (that is, every object that is mail-enabled). This filter is the filter for the default recipient policy.

If you create additional recipient policies, the filters may become much more complex as you restrict the objects that the policies apply to. Exchange provides a user interface that builds filters for the administrator based on common-sense criteria, instead of requiring you to construct filters manually.

For example, the following filter is automatically generated to capture all Exchange users, contacts, and groups: To construct this filter on a policy:

1. Start Exchange System Manager.
2. Expand Recipients, expand Recipient Policies, and the either open the properties of the policy that you want to edit or create a new policy.
3. On the General tab, click Modify.
4. Click to select all the check boxes on the General tab.
5. Click the Storage tab, and then click Mailboxes on any server.
   
   You do not have to configure anything on the Advanced tab.

To modify this filter so that it checks for the existence of the givenName and the sn attributes:

1. Open the properties of the recipient policy, and then click Modify.
2. Click the Advanced tab, select the First Name user field and the Last Name user field, and then set the condition for these fields to Present.

After you complete these two procedures, the LDAP filter appears as the following filter: Notice that (objectCategory=user)(givenName=*)(sn=*) has been added near the end of the filter. This filter now captures only users who have both a givenName and an sn attribute. This filter ignores distribution lists, contacts, and Exchange system objects.

Finding and Removing Duplicate Addresses

Duplicate addresses that are created by auto-naming specifiers typically follow a predictable pattern (for example, _12345@domain.com, _12346@domain.com, and similar addresses). Therefore, you can perform an automated search for these addresses and perform automated cleanup.

You can use the Ldifde.exe utility that is included in Windows 2000 to export and import Active Directory information as plain text in the LDIF format. Ldifde uses the same standard LDAP search syntax as the Exchange recipient policy filters. You can paste a recipient policy filter into an Ldifde command line and create a text file that lists every object in a container that is captured by the filter. The following command is an example of the general syntax to create a text file: In this command, all objects in Subdomain that fit the filter are written to File.txt. To capture objects from the Configuration container, you must specify the container and the first domain that is installed in the forest, for example: The -l parameter restricts the attributes for the object that are written to File.txt. If you omit -l, all attributes for each object are listed. To list none of the attributes, use -l nothing. If you do so, the command exports only the distinguished name of each object to File.txt.

To find all duplicate or multiple proxy addresses, you can run Ldifde against every domain in your organization and against the Configuration container for your forest. To do so, you must define a unique search pattern that matches only the unwanted addresses.

For this example, the search pattern is *SMTP:_*@*. This search pattern captures every SMTP (Internet Mail) Exchange e-mail proxy address that starts with an underscore. For example, run the following command: This command exports data to Badproxies.txt that is similar to the following data: The Badproxies.txt file contains records for objects whose proxy addresses include an SMTP address that begins with an underscore. All other addresses for these objects are also exported. You can filter out these extra addresses later.

Some addresses that begin with underscores may be legitimate. If this condition is true, you must remove those objects from the Badproxies.txt file before you continue. Run the following Ldifde command to identify these objects: This command searches for objects whose first name, last name, Windows logon name, Exchange alias, or display name starts with an underscore. You can adapt the search filter to the pattern that is appropriate for your particular environment.

Note If the syntax of your Ldifde command is incorrect, Ldifde may not return an error. Instead, you may not see any search results at all. If your search returns no objects, test the Ldifde command by altering the search filter in a minor way to return results. For example, replace the underscore with the letter A.

Also, if you do not have permissions to view objects or attributes, these items are not returned by the search and you do not receive an error that states that you do not have permissions for certain object. When you search a domain container, Microsoft recommends that you log on as a domain administrator. When you search the Configuration container, Microsoft recommends that you log on as an enterprise administrator.

After you verify that Badproxies.txt contains only objects that have e-mail addresses that you want to remove, you must filter and re-format the LDIF file so that you can import it again. The LDIF import file format and the export file format differ significantly.

You can use the following sample script to convert and filter the import file. This script is provided as a sample only; use or modify this script at your own risk. This script in itself cannot change Active Directory information. You must manually import the output file from the script to Active Directory to make changes. Each line in the script is double-spaced to make it easier to detect improper line wrapping. To run the script, copy and paste the following text to a plain text file, and then name this file Proxyfix.bat. Notes

• The output file that this script generates cannot be imported into Active Directory before you edit it first. Depending on how you run the script, the script may generate a file that deletes every proxy address in the whole domain. You must examine the output file closely before you apply it to Active Directory to make sure that it does what you want.
• The top three lines in the file are a comment and white space that you must remove before you run the file. If there are warnings at the top of the file, you must remove these also.
• If the script pauses with the "Broken line encountered!" message, you must correct the line that is reported and run the script again.
• The LDIF standard divides long lines into multiple lines by continuing the line after a line break with a space in the first column. The sample script does not account for this standard, except to report when this kind of line is found. If there are many broken lines in the file, you can correct these lines by using a global find and replace procedure. To do so, use a text editor that supports find and replace across line breaks. You can use Microsoft Word to perform a global find and replace, but you must remember to save the edited file as plain text. In Word, you can search for the following item, and replace it with nothing:
  ^p[space character]
  Note that the search text is the caret character (^) and a lowercase p, not CTRL+P.
• If a record in the output file causes all proxy addresses to be deleted from the object, the script tries to insert a warning at the top of the file. If you use this script to generate LDIF import files, you must closely examine these files to make sure that they contain only the records that you want to be imported.
  
  The following record format deletes all proxy addresses:

  dn: CN=OBJECT,CN=CONTAINER,....,DC=com
  changetype: modify
  delete: proxyAddresses
  -
  							

  If no specific proxy address is named in the record, all addresses are deleted. The following record format deletes only a single proxy address:

  dn: CN=OBJECT,CN=CONTAINER,....,DC=com
  changetype: modify
  delete: proxyAddresses
  proxyAddresses: SMTP:OBJECT@domain.com
  -
  						

• To back out of changes that are made by the output file (except for records that perform wholesale deletions), search the output file, replace "delete: proxyAddresses" with "add: proxyAddresses," and then import the file again.
• To export all proxy addresses for all objects in a container, use the -R "(proxyAddresses=*)" Ldifde search filter.
  
  You can use this file as a backup file of proxy addresses that exist at a specific point in time. If you process this file through Proxyfix.bat, this file can be created as an import file that you can use to restore addresses if necessary.
• If you do not define a filter as the third parameter on the Proxyfix.bat command line, all proxy addresses from the input file are copied to the output file. You can change "delete: proxyAddresses" to "add: proxyAddresses" to merge addresses with addresses that are already present in Active Directory. You can also use "replace: proxyAddresses" to remove all proxy addresses from Active Directory and replace them with only the addresses listed in the output file.
• If you deleted all the proxy addresses from all the objects, the objects are changed and the Recipient Update Service is triggered to restore all addresses according to current recipient policies. However, before this process is complete, non-delivery reports (NDRs) for in-transit mail may be generated. If addresses in the Configuration container are deleted, Exchange services may stop or you may not be able to start them if they are already stopped. Microsoft recommends that you use extreme care when you handle proxy addresses, especially for objects in the Configuration container.

To import an LDIF file, run the following command: