Replication USN semantics around Azure AD Connect

[Andrew Bartlett]

On Wed, 2023-06-28 at 10:15 +0200, Stefan Metzmacher wrote:
> Hi Andrew,
> > I've been working to make Azure AD connect work with Samba.
> > This is working for small domains, but for larger domains we hit
> > aproblem.
> > Our USN handling in GetNCChanges can sometimes return the
> > sametmp_hightest_usn, so we have this code:
> > 			/*			 * We need to make sure
> > that we never return the			 * same highwatermark
> > within the same replication			 * cycle more than
> > once. Otherwise we cannot detect			 * when the
> > client uses an unexptected highwatermark.			 *		
> > 	 * This is a HACK which is needed because our			
> >  * object ordering is wrong and set tmp_highest_usn			
> >  * to a value that is higher than what we already			
> >  * sent to the client (destination dsa).			 */		
> > 	r->out.ctr->ctr6.new_highwatermark.reserved_usn += 1;
> > 
> > To make a new higher USN for this case.
> > However, Azure AD connect comes back only with the tmp_highest_usn,
> > egreserved_usn is always zero.  This means we declare the client-
> > suppliedhighwatermark as being older, and start replication from
> > scratch.
> > Starting from zero then means we give the same data, end in the
> > sameplace and the client has an infinite loop against us.
> > Do you have any clues, other than a revamp of our replication
> > logic, tofix this both sort-term and to remember for the longer
> > term?
> > In the meantime I've suggested 'drs max objects sync = 100000' and
> > 'drsmax link sync = 100000' in the smb.conf to avoid multiple
> > chunks.
> > Thanks for any clues,
> 
> I think I hit some similar when migrating large samba domains
> toWindows 2008R2. But I don't remember it 100%.
> Currently we're using these branches to avoid it:
> https://git.samba.org/?p=metze/samba/wip.git;a=shortlog;h=refs/heads/v4-10-drsuapi
> https://git.samba.org/?p=metze/samba/wip.git;a=shortlog;h=refs/heads/v4-13-drsuapi
> 
> They are also in 
> https://git.samba.org/?p=metze/samba/wip.git;a=shortlog;h=refs/heads/master-drsuapi,as
> the top 16 commits, but that also contains a lot of unfinished
> unrelated stuff, e.g. implementing timed group memberships.

Thanks.  I've taken a look over those, but I don't see how it would
impact on the USN or the highwatermark.  
Am I missing something in those changes that would help ensure the USN
keeps moving or could this be a different loop?
Andrew Bartlett

-- 
Andrew Bartlett (he/him)       https://samba.org/~abartlet/
Samba Team Member (since 2001) https://samba.org
Samba Team Lead                https://catalyst.net.nz/services/samba
Catalyst.Net Ltd


Proudly developing Samba for Catalyst.Net Ltd - a Catalyst IT group
company

Samba Development and Support: https://catalyst.net.nz/services/samba

Catalyst IT - Expert Open Source Solutions