Fixing lingering object problems in complex Active Directory forests
Lingering objects can provide an inconsistent view of the Active Directory and confuse administrators. Learn how to find and remove them in complex forests.
A lingering object is any Active Directory object that has been deleted but gets reanimated when a domain controller (DC) does not replicate the change during the domain's tombstone lifetime period. Objects can be present on one DC but not on another. This provides an inconsistent view of the Active Directory and often confuses administrators.
The following case study demonstrates how to find and remove lingering objects in a complex forest. The example features a large global forest with an empty parent root domain and three child domains that represent geographies of the world. The child domains are called the Americas, Europe, and Asia and the parent root domain is Corp.com.
In this scenario, when the user account "Ellie" was created, it generated a conflicting (CNF) object; while Ellie's account worked in the Europe domain, there also was a CNF version of the object. Typically, CNF objects end up in the Lost and Found container and can be deleted. However, this was not the case in this example.
If this occurs, you can use the Find function in the Active Directory Users and Computers (ADUC) snap-in to find the object – just make sure Entire Directory is set as the location.
To specify the location, right-click on the domain, select Find, and in the In: drop-down box, select Entire Directory, as shown in Figure 1.
Figure 1: Find object dialog box (click to enlarge)
In this case study, the search results (Figure 2) showed that the object was in the Europe.corp.com domain.
Figure 2: Search results (click to enlarge)
Therefore, from the search we learned:
- The Guid is 9abcee2e-aff2-459e-b6fd-e4a0ae58f39d
- The object is in the Europe.corp.com domain
Other methods that could have been used to find the object include:
- The ADFind command:
adfind -h GCServerName -gcb -f name=*lichfield* -dn
GCServerName is the name of a Global Catalog. - The LDIFDE command:
Ldifde -f user.ldf -d "dc=europe,dc=Corp,dc=com" -r "(&(objectClass=user)(name=*lichfield*))"
In some cases, it may be possible to delete the object with the free Admod tool from Joeware.net:
However, the tool did not work in this scenario.
In this case, the next step is to find the DCs the object exists on. Since not all DCs can find the object, use the Repadmin /showobjmeta command to get a list of DCs that know about this object.
The command uses the following GUID:
The * is the DC list (this command runs on every DC).
This command searches for the object of the specified GUID. If it finds it, it dumps all the attributes; if the object is not found, the DC returns the following error:
DsReplicaGetInfo() failed with status 8333 (0x208d):
Directory object not found.
You can dump the output to a file, then go through the file manually or with findstr and look for some part of the above error message. Eliminate those DCs.
Regardless of the method, in the end you will have a list of DCs with the object.
Using the Repadmin/RemoveLingeringObjects command
The next step is the Repadmain /RemoveLingeringObjects command.
The syntax of the command is:
Repadmin/removeLingeringObjects <Dest_DC_LIST> <Source DC GUID> <NC> [/ADVISORY_MODE]
Where:
- The Dest_DC_List is the one you just made with the /showobjmeta command
- The Source DC GUID is the DC object GUID of a DC in the domain that has the object and that is up to date in replication. In this case I picked the PDCe.
- The NC is the naming context to operate on. Use the DN
The command used in this case study is:
There are some "gotchas" here. First, this command has to run for every DC in every domain in the forest. In this scenario, the command was run four times for DC1. Note that the >> operator was used to append all these to a single log file since it was scripted. This made a nice log file later.
Repadmin /removelingeringobjects eu-dc1 9abcee2e-aff2-459e-b6fd-e4a0ae58f39d
DC=europe,DC=Corp,DC=com >> error.log
Repadmin /removelingeringobjects eu-dc1 9abcee2e-aff2-459e-b6fd-e4a0ae58f39d
DC=americas,DC=Corp,DC=com >> error.log
Repadmin /removelingeringobjects eu-dc1 9abcee2e-aff2-459e-b6fd-e4a0ae58f39d
DC=Asia,DC=Corp,DC=com >> error.log
Repadmin /removelingeringobjects eu-dc1 9abcee2e-aff2-459e-b6fd-e4a0ae58f39d
DC=Europe,DC=Corp,DC=com >> error.log
Since I wasn't sure that the primary domain controller (PDC) was a good source, I ran the file for each of the DCs in the Europe domain:
Repadmin /removelingeringobjects eu-dc1 b4951358-e44e-41c7-bf86-f2ff1184c66b
DC=europe,DC=corp,DC=com >> error.log
Repadmin /removelingeringobjects eu-dc1 b4951358-e44e-41c7-bf86-
f2ff1184c66bDC=americas,DC=corp,DC=com >> error.log
Repadmin /removelingeringobjects eu-dc1 b4951358-e44e-41c7-bf86-f2ff1184c66b
DC=Asia,DC=corp,DC=com >> error.log
A script was written to create a batch file and avoid lots of typing.
Setting strict replication consistency
To see if the object was removed, run the Repadmin/showobjmeta command and the FIND operation in the ADUC snap-in. In this case study, the object was deleted -- but it reappeared three days later.
This meant the object was replicating ahead of the script. Therefore, all the objects weren't caught and the objects eventually replicated back. Since this was a domain upgraded from Windows 2000, the strict replication consistency registry value was set to "loose":
Data Type = Reg_DWORD
Value Data=0
This permits lingering objects to replicate to other DCs. The registry value was set to 1 (strict) and the script was rerun. This successfully deleted the object.
The registry value can be set easily with the following command:
There is only one well-known key, "strict". The DCList can be specified as *.corp.com to include all DCs in the domain, although this typically only needs to be executed on Global Catalogs.
The following is an example of this command and the output. I specified the forest root domain in the command in my test forest, and it operated on DCs in two child domains plus the parent. DCs include Windows Server 2003, 2008 and 2008 R2.
C:\>repadmin /regkey *.wtec.adapps.hp.com +strict
repadmin running command /regkey against server WTEC-DC2.Wtec.adapps.hp.com
DWORD Regkey Value: strict : 1
DWORD Regkey Value: strict : 1
repadmin running command /regkey against server w2k8r2-dc1.w2k8r2.Wtec.adapps.hp.com
DWORD Regkey Value: strict : 1
DWORD Regkey Value: strict : 1
repadmin running command /regkey against server w2k8-dc1.w2k8.Wtec.adapps.hp.com
DWORD Regkey Value: strict : 1
DWORD Regkey Value: strict : 1
Successful running of this command ensures that the strict replication consistency registry value is set to 1 (strict). This prevents lingering objects from replicating between DCs. If you want to enable loose behavior (which is sometimes required in troubleshooting), set the value to -strict:
C:\>repadmin /regkey wtec-dc1.wtec.adapps.hp.com –strict
New Regkey Value: strict does not exist
When –strict is used, the strict replication consistency value is removed from the registry. The first message states the current setting (set to 1). Here the output states that "strict does not exist." The second message shows it has been deleted. Running+strict on WTEC-DC1 produces this message:
C:\>repadmin /regkey wtec-dc1.wtec.adapps.hp.com +strict
The initial value of the key is "does not exist", which means it is set to "loose" consistency and will replicate lingering objects. The second message shows it was set to 1 or strict consistency. Therefore, you can run the command and see the current state of this key.
In the initial example, both messages stated the value was strict:1. This means the key was set to strict already, and it set it to 1.
Using LDIFDE to remove lingering objects
In addition to the Repadmin /RemoveLingeringObjects command, it is possible to remove lingering objects with LDIFDE. It is a more complicated method because LDIFDE input files need to be imported in order to make attribute changes.
The LDIFDE command is:
ldifde -i -f %InputFile% -s
The –f option indicates the name of the input (import) file.
The results can be outputted to a log file. In this example, >> was used to append it all to a single file. The input file would look like this:
File name: DeleteLO.ldf
--------------------------------- dn:
changetype: modify
replace: RemoveLingeringObject
RemoveLingeringObject: CN=NTDS Settings,CN=EU-DC1,CN=Servers,CN=Paris, CN=Sites,CN=Configuration,dc=europe,dc=corp,dc=com,<GUID=9abcee2e-aff2-459e-b6fd-e4a0ae58f39d>
-------------------------------
Remember, in an LDIFDE import file there must be a blank space at the end of the instructions, followed by a line with a hyphen. The batch file used in this case study looked like this:
@ECHO off
SET InputFile=DeleteLO.ldf
ldifde -i -f %LDFFile% -s Eu-DC1.europe.corp.com >> error.log
ldifde -i -f %LDFFile% -s EU-DC2.europe.corp.com >> error.log
Of course, an entry is needed for each domain controller in the forest. After running the above script, examine the error log to determine if it was successful on all machines.
While lingering object detection and removal can be frustrating – especially when it comes to finding the object itself – using these basic techniques with the Repadmin tool will help you successfully clean lingering objects from the Active Directory.
For more information on lingering objects in a Windows 2000 or 2003-based forest, check out the following Microsoft articles:
- A lingering object is detected (Event ID 1388 or 1988)
- Lingering objects may remain after you bring an out-of-date global catalog server back online
More AD troubleshooting tips from Gary Olsen
- How to find and remove lingering objects in A
- Troubleshooting poor client logon performance
- Debugging Active Directory database errors
ABOUT THE AUTHOR
Gary Olsen is a systems software engineer for Hewlett-Packard in Global Solutions Engineering. He authored Windows 2000: Active Directory Design and Deployment and co-authored Windows Server 2003 on HP ProLiant Servers. Gary is a Microsoft MVP for Directory Services and formerly for Windows File Systems.
