This week i’ve set up the first Windows 2012 R2 domain controller at a customer. All worked good and looked fine. But when i had to create two new user accounts i found out that these two accounts weren’t replicated to the new domain controller, i’ve just set up a day erlier, nor to another domain controller in another site. I discovered also that the NTDS settings and replication topology wasn’t complete. The new domain controller had not a single connection to active directory domain services. The customer has two sites which are connected over a leased line. Both sites have their domain controllers. Those new user accounts i’ve created on an existing Windows 2008 R2 domain controller.
After nearly two days of testing and troubleshooting the problem seems to be solved. All domain controllers are replicating and talking with each other domain controllers. When i create a user account it will show up instantly on all other domain controllers. Also the replication topology is now looking good. KCC generated the missing topology now automatically, which wasn’t the case directly after the new domain controller was on duty.
I want to provide you some information about this issue and how i solved it. Probably it will help you solving your Active Directory replication issues. And if not i hope it will be at least something you can check if this patricular thing is ok and help you with troubleshooting.
Initial problem / gathering facts
I tested the replication with some dummy users. On each domain controller i created a user account with a specific name. So i was able to identify the user from which domain controller it came originally. The result looked like this:
In Site A the new domain controller (named DC03 / Server 2012 R2) was able to replicate to all other domain controllers on both sites. Also in Site A the two existing domain controllers (named DC01 and DC02 / both Server 2008 R2) were only able to replicate between them self. In Site B the existing domain controller (named DC04 / Server 2008 R2) was only able to replicate to DC01 and DC02 in Site A.
First steps of troubleshooting
First i checked DNS. All DNS records on this Active Directory were correct. They were resolvable to an IP address, and also backwards, the IP addresses were resolvable to a DNS record. So DNS wasn’t the cause of this problem.
Checking Active Directory
I continued to dig. One of the first checks was with dcdiag to see if there is any issues, and probably to find out what’s the issue. This test told me that the replication topology was not complete. And this lead me to dig deeper into Active Directory Sites and Services to look if there is all good. I wanted to make sure that every site has its domain controllers and that the replication topology was in order. This was the first location were i discovered some issues. The new domain controller did not have any automatic generated replication connections under NTDS settings. The existing domain controllers did not have any automatic generated replication connections to the new domain controller, but for them selfs. How can i fix this?
With repadmin /kcc you can force KCC (Knowledge Consistency Checker) to run on a domain controller. KCC will run and check its inbound replication topology immediately and will generate missing connections. I did that. But it didn’t help. No new generated connections. So grab the shovel and let’s dig again few inches…
The next check i did was to find out if any specific role probably has been moved to another domain controller, which shouldn’t be moved. With netdom query fsmo you will find out which domain controller is holding the five (or some of the five if you distributed them) FSMO (Flexible Single Master Operations) roles:
- Schema master
- Domain naming master
- RID master
- PDC emulator
- Infrastructure master
All of the above roles were holded by the first domain controller (DC01 / Server 2008 R2). So this couldn’t be the cause of this replication issue.
Lucky me i found out (and that was more luck than other) that one special role was moved to the new domain controller (DC03 / Server 2012 R2). You probably know this role. It was the ISTG (Infrastructure Topology Generator) role. I didn’t know why this role is on another server than on the first domain controller DC01. And for sure i didn’t move this role.
The first domain controller promoted in a site takes on the ISTG role. And this is a fairly sticky role. If the current ISTG becomes unavailable for 60 minutes, an election is held by the other DCs in the site to appoint a new ISTG. This can sometimes cause problems for Active Directory replication. And i think this was actually causing problems with the customers Active Directory replication. What makes me wondering is the fact of the election by the other DCs in the site. The former holder of the ISTG role wasn’t unavailable. I created users on it and found out that replication does not work proper. What the heck is going on?
Moving ISTG role back to the origin
Everytime i read about ADSIEdit my spider senses are tingling. Normaly you shouldn’t have to do anything in ADSIEdit, if anything is running fine. IF anything is running fine. In this case i had to. A short search led me to the EXPTA blog of Jeff Guillet. I had to move the ISTG role back to the origin server, which was DC01. And this is only possible through ADSIEdit. And because i knew that the newly installed DC03 is replicating fine to all other domain controllers i did this change on this domain controller.
- Open ADSIEDIT.msc
- Expand Configuration [DomainController].
- Expand CN=Configuration,DC=<domain>,DC=<com>.
- Expand CN=Sites.
- Highlight CN=<sitename> for the site where you want to change the ISTG Server.
- In the details pane, right-click on CN=NTDS Site Settings and select Properties.
- Locate the interSiteTopologyGenerator attribute and you will see which Domain Controller is designated as the ISTG server.
- To change the server, click Edit and then change the server name, as shown below.
After waiting some minutes i checked this specific setting on all other domain controllers and saw that it was replicated correctly. At least one good sign? But let’s move on…
Re-enable automatic topology generator
In Active Directory Sites and Services i found out that on each site the automatic generation of replication topology was disabled. I don’t know why, but probably because there was a replication issue I can’t explain. So again back to ADSIEdit and checking the values. I searched the internet and found a helpful ressource on Isaac Oben’s blog.
ISTG reference numbers:
|1||disable automatic intrasite topology generation|
|16||disable automatic intersite topology generation|
|17||disable both intrasite and inter-site topology generation|
- Start ADSIEdit
- Connect to a domain controller and expand the configuration tab
- Expand CN=Configuration,CN=YourDomain,CN=com
- Expand CN=Sites
- Select CN=[name of the site to configure]
On the right panel right click the “CN=NTDS Site Settings” and choose properties.
Scroll down to the “options” value and change it to one of the above values. To re-enable the topology generator you have to enter the value 0.
I did this first on the newly installed domain controller because he was replicating fine to any other DC. After waiting some minutes i changed this value for all sites and verified that these settings were replicated to all other DCs. I had to manually adjust this on some of the other DCs.
But now automatic topology generator was re-enabled.
After lunch i checked all DCs again with dcdiag and checked if my previously created dummy users were replicated to all DCs. And they were.
I didn’t see this kind of issue before. There were other replication issues, sure, but not one of this kind. And that even after promoting a new domain controller. Finally adjusting the ISTG role and re-enabling the topology generator did the trick.