Troubleshooting AD Replication error 8333: Directory Object Not Found

Symptoms

This article describes the symptoms, cause, and resolution steps when Active Directory replication fails with error 8333: Directory object not found (ERROR_DS_OBJ_NOT_FOUND)

1. Possible formats for the error include:

Decimal

Hex

Symbolic

Error string

8333

0x208d

ERROR_DS_OBJ_NOT_FOUND

Directory object not found.


2. The following events could be logged
 

Event Source

Event ID

Event String

NTDS Replication

2108

This event contains REPAIR PROCEDURES for the 1084 event which has previously been logged. This message indicates a specific issue with the consistency of the Active Directory database on this replication destination. A database error occurred while applying replicated changes to the following object. The database had unexpected contents, preventing the change from being made.

Object:

OU=TestOU,DC=contoso,DC=com

Object GUID:

1284b336-6e2a-4b80-86ce-d48f558e9aa2

Source domain controller:

A52b57e3-92b9-4264-822b-72963eaf1030._msdcs.contoso.com

 

Additional Data

Primary Error value:

8333 Directory object not found.

Secondary Error value:

-1601 JET_errRecordNotFound, The key was not found


NTDS General

2031

The DS Service Configuration object is not found. It might have been accidentally deleted. The Active Directory will be able to operate normally, but you will not be able to set certain service parameters, such as LDAP limits, default query policies, and SPN mappings.

DS Service Configuration object: CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=contoso,DC=com

Error: 8333 (Directory object not found.)

User Action: Try to restore the DS Service Configuration object.

3. There may be output from “repadmin /replsum”

DC-1-03 03h:14m:11s 1 / 52      1 (8333) Directory object not found.
DC-2-01 03h:13m:39s 1 / 26      3 (8333) Directory object not found.
DC-3-09 03h:08m:45s 2 / 103    1 (8333) Directory object not found.
DC-4-03 03h:05m:52s 1 / 13      7 (8333) Directory object not found.

4. DCPromo may fail while promoting a new domain controller and you will see the following errors in the DCPROMO log

12/15 11:01:44 [INFO] Creating new domain users, groups, and computer objects
12/15 11:01:44 [INFO] Error - Active Directory is missing critical information after installation and cannot continue. If this is a replica domain controller, rejoin this server to the domain. (8333)
12/15 11:01:45 [INFO] NtdsInstall for contoso.com returned 8333
12/15 11:01:45 [INFO] DsRolepInstallDs returned 8333
12/15 11:01:45 [ERROR] Failed to install to Directory Service (8333)

NOTE: Error 8333 translates to ERROR_DS_OBJ_NOT_FOUND or "Directory object not found."

5. While trying to re-host a partition on the Global catalog

repadmin /rehost <dc-name> <partition to rehost> <good source>
repadmin /rehost failed with DsReplicaAdd failed with status 8333 (0x208d)

Cause

The error status 8333 “Directory Object Not Found” has multiple root causes including:

1. Database corruption with additional associated errors logged in the event log of the source domain controller:

 SourceEvent IdDescription
NTDS Replication 2108 This event contains REPAIR PROCEDURES for the 1084 event which has previously been logged. This message indicates a specific issue with the consistency of the Active Directory Domain Services database on this replication destination. A database error occurred while applying replicated changes to the following object. The database had unexpected contents, preventing the change from being made.

Object: CN=chduffey,OU=IT,OU=Corp,DC=contoso,DC=com
Object GUID: 13557897-e45d-4af6-8f25-15b306d6e927
Source domain controller: c4efaf4e-d652-4630-8623-afec5ebc8532._msdcs.contso.com

Additional Data
Primary Error value: 8333 Directory Object Not Found.

NTDS General 1168 Error -1073741790(c0000022) has occurred (Internal ID 3000b3a). Please contact Microsoft Product Support Services for assistance.
 

Microsoft-Windows-
ActiveDirectory_DomainService

 1084Internal event: Active Directory could not update the following object with changes received from the following source domain controller. This is because an error occurred during the application of the changes to Active Directory on the domain controller.
 NTDS Replication 1699The local domain controller failed to retrieve the changes requested for the following directory partition. As a result, it was unable to send the change requests to the domain controller at the following network address. 8446 The replication operation failed to allocate memory

Additionally you may see replication status code:

 CodeSources Additional Infortmation 
 8451Repadmin, DcPromo, as sub-code in Database Corruption Events Refer to the troubleshooting guide for 8451 in the first instance if this error is identified.

2645996

2. Lingering Objects with associated errors logged:

 SourceEvent IDDescription
NTDS Replication1988 Active Directory Replication encountered the existence of objects in the following partition that have been deleted from the local domain controllers (DCs) Active Directory database. Not all direct or transitive replication partners replicated in the deletion before the tombstone lifetime number of days passed. Objects that have been deleted and garbage collected from an Active Directory partition but still exist in the writable partitions of other DCs in the same domain, or read-only partitions of global catalog servers in other domains in the forest are known as "lingering objects".
NTDS Replication1388Another domain controller (DC) has attempted to replicate into this DC an object which is not present in the local Active Directory database. The object may have been deleted and already garbage collected (a tombstone lifetime or more has past since the object was deleted) on this DC. The attribute set included in the update request is not sufficient to create the object. The object will be re-requested with a full attribute set and re-created on this DC.

Additionally you may see the following replication status codes:

Source SourcesDescription
8606Repadmin, DCPromo, sub code in NTDS Replication events 

Refer to the troubleshooting guide for 8606 in the first instance if this error is identified.

2028495

1722Repadmin, DCPromo, sub code in NTDS Replication events

Refer to the troubleshooting guide for 1722 in the first instance if this error is identified.

2102154

3. Conflict Objects.

4. 3rd Party process

a. Antivirus
b. Directory synchronisation software

Resolution

Investigation of the 8333 "Directory Object Not Found" error message should begin on the source domain controller in the replication partnership. Referring to each of the possible causes of the issue from the "cause" section of this document, a support professional should begin their investigation on the source of the source/destination replication partnership.

1. Check for indications of Active Directory (JET) Database corruption:

a. Review the Directory Services event log on the source and destination replication partners for JET database corruption events. Possible events include:

 SourceEvent IdDescription
NTDS Replication 2108 This event contains REPAIR PROCEDURES for the 1084 event which has previously been logged. This message indicates a specific issue with the consistency of the Active Directory Domain Services database on this replication destination. A database error occurred while applying replicated changes to the following object. The database had unexpected contents, preventing the change from being made.

Object: CN=chduffey,OU=IT,OU=Corp,DC=contoso,DC=com
Object GUID: 13557897-e45d-4af6-8f25-15b306d6e927
Source domain controller: c4efaf4e-d652-4630-8623-afec5ebc8532._msdcs.contso.com

Additional Data
Primary Error value: 8333 Directory Object Not Found.

NTDS General 1168 Error -1073741790(c0000022) has occurred (Internal ID 3000b3a). Please contact Microsoft Product Support Services for assistance.
 

Microsoft-Windows-
ActiveDirectory_DomainService

 1084Internal event: Active Directory could not update the following object with changes received from the following source domain controller. This is because an error occurred during the application of the changes to Active Directory on the domain controller.
 NTDS Replication 1699The local domain controller failed to retrieve the changes requested for the following directory partition. As a result, it was unable to send the change requests to the domain controller at the following network address. 8446 The replication operation failed to allocate memory

Additionally you may see replication status code:

 CodeSources Additional Infortmation 
 8451Repadmin, DcPromo, as sub-code in Database Corruption Events Refer to the troubleshooting guide for 8451 in the first instance if this error is identified.

2645996

b. Enable advanced directory services replication logging:

Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:

322756 How to back up and restore the registry in Windows

To increase NTDS diagnostic logging, change the following REG_DWORD values in the registry of the destination domain controller under the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTDS\Diagnostics

Set the value of the following subkeys to 5:

5 Replication Events
9 Internal Processing

Note Level 5 logging is extremely verbose and the values of both sub-keys should be set back to the default of 0 after the problem is resolved. Filtering the Directory Services event log should be performed to isolate and identify these events.

c. Review the event logs for the new events that were generate from the increased logging for error values that will give a definitive view of the Database Corruption.

d. If database corruption has been detected, ensure that recent backups exist of each domain in the forest.

e. Restart the domain controller reporting the database corruption in directory services restore mode. (Press F8 while the server is restarting or if this is not possible open msconfig.exe and choose “Active Directory Repair” in the “boot” options.).

f. To perform an inspection of the database in Directory Services Restore Mode:

i. Open a command prompt
ii. Type “ntdsutil”
iii. Type “activate instance ntds”
iv. Type “Semantic database analysis”
v. Type “go”

If errors are detected they will be displayed to the console and written to a log file in the current working directory.

g. If database corruption errors are detected, you are advised to contact Microsoft Support Services.

h. As a last option. You can demote the domain controller, and promote it again to replace the database and replicate the contents from another server in the domain.

Note: If an Active Directory database has been corrupted in you environment it is important to consider the source of the corruption to avoid issues in the future. Some of the known causes of such corruption are:

i. Failing Hardware: Hard Disk or controller
ii. Caching: Hard Disk controller
iii. Out-dated Drivers: Hard Disk controller
iv. Out-dated Firmware: BIOS, Hard Disk controller, Hard Disk
v. Sudden power Loss 

2. Check for the existence of and remove Lingering Objects on all domain controllers in the forest.

There are multiple approaches to check for Lingering Objects including:

a. Check for the existence of the following Directory Services events on domain controllers in the forest:

 SourceEvent IDDescription
NTDS Replication1988 Active Directory Replication encountered the existence of objects in the following partition that have been deleted from the local domain controllers (DCs) Active Directory database. Not all direct or transitive replication partners replicated in the deletion before the tombstone lifetime number of days passed. Objects that have been deleted and garbage collected from an Active Directory partition but still exist in the writable partitions of other DCs in the same domain, or read-only partitions of global catalog servers in other domains in the forest are known as "lingering objects".
NTDS Replication1388Another domain controller (DC) has attempted to replicate into this DC an object which is not present in the local Active Directory database. The object may have been deleted and already garbage collected (a tombstone lifetime or more has past since the object was deleted) on this DC. The attribute set included in the update request is not sufficient to create the object. The object will be re-requested with a full attribute set and re-created on this DC.

Additionally you may see the following replication status codes:

 CodeSources Additional Infortmation 
 8451Repadmin, DcPromo, as sub-code in Database Corruption Events Refer to the troubleshooting guide for 8451 in the first instance if this error is identified.

2645996

b. Use repldiag.exe to examine the forest for lingering objects.

Repldiag may be downloaded from codeplex.com. To perform the lingering object check in advisory mode use the syntax:

repldiag /RemoveLingeringObjects /AdvisoryMode

Directory Service event 1942 will be logged on each domain controller and will indicate the number of lingering objects that were detected in each directory partition.

The work being performed by repldiag may also be performed with the built in directory services replication tool: Repadmin.exe.

For support professionals preferring to use repadmin.exe, the partial command will be Repadmin /removelingeringobjects. Repldiag.exe provides an advantage over Repadmin.exe in that it can be used to search all directory partitions, on all servers in the forest with a single command.

If Lingering objects are detected:

a. Perform a system state backup of two domain controllers in each domain in the forest.
b. Use repldiag.exe to perform clean-up of lingering objects:

repldiag /RemoveLingeringObjects

c. Each domain controller will log a directory services event 1942 for each directory services partition to indicate if lingering objects have been removed.
For an alternate approach to the removal of lingering objects you can use the built in tool Repadmin.exe with the /removelingeringobjects switch. This approach requires multiple commands, repldiag provides an aggregate of the commands Repadmin.exe would use. 

 
3. Check for the existence of and remove conflict objects:

a. Search the relevant directory partitions for CNF managed objects and the object that the conflict-mangled object conflicted with the following syntax:

repadmin /showattr localhost "dc=parent,dc=com" /subtree /filter:"((&(objectClass=*)(cn=*\0acnf:*)))" /atts:objectclass,whencreated,whenchanged

In this example “dc=parent,dc=com” is the distinguished name for the parent.com domain.

In most circumstances the 8333 error will indicate which directory partition(s) should be evaluated for conflict objects. It is recommended that the configuration partition is checked in all instances:

repadmin /showattr localhost "cn=configuration,dc=parent,dc=com" /subtree /filter:"((&(objectClass=*)(cn=*\0acnf:*)))" /atts:objectclass,whencreated,whenchanged

b. Review the attributes, attribute values and if present, subordinate objects to determine which object should remain and which should be deleted
c. Ensure you have an up to date backup of the directory
d. Delete the conflict mangled object / container or the object it conflicted with using LDP.EXE, ADSIEDIT or one of the Active Directory management tools.

4. Perform testing of the replication partners with third party components removed.

Multiple third party products have been found to cause this issue including:

a. Anti-Virus software
b. Directory Synchronisation

內容

文章識別碼:2703708 - 最後檢閱時間:2017年2月1日 - 修訂: 4

意見反應