[Samba] Azure AD Connect and replication issues

Thu Oct 22 15:40:34 UTC 2020

just small update:

- idfix tool (Directory Synchronization Error Remediation Tool / 
https://github.com/microsoft/idfix) shows just small issues like 
empty/missing displayName attrib in some of objects which I have 
corrected and no more issues present at all.
- no errors from AAD connect event viewer: final log message is 
"Scheduler::SchedulerThreadMain : Completed configured scheduler 
operations." (no errors before)
- aad connect info:

Azure AD Tenant - Settings

PasswordHashSync: True
ForcePasswordChangeOnLogOn: False
UserWriteback: False
DeviceWriteback: False
UnifiedGroupWriteback: False


Azure AD Connect - Global Settings

Microsoft.AADFilter.ApplicationList:
Microsoft.AADFilter.AttributeExclusionList:
Microsoft.Configuration.ImportDate: 2020-10-19 17:08:19Z
Microsoft.ConnectDirectories.WizardDirectoryMode: AD
Microsoft.DeviceWriteBack.Container:
Microsoft.DeviceWriteBack.Forest:
Microsoft.DirectoryExtension.SourceTargetAttributesMap:
Microsoft.OptionalFeature.AutoUpgradeState: Enabled
Microsoft.OptionalFeature.AutoUpgradeSuspensionReason:
Microsoft.OptionalFeature.DeviceWriteBack: False
Microsoft.OptionalFeature.DeviceWriteUp: True
Microsoft.OptionalFeature.DirectoryExtension: True
Microsoft.OptionalFeature.DirectoryExtensionAttributes:
Microsoft.OptionalFeature.ExchangeMailPublicFolder: False
Microsoft.OptionalFeature.ExportDeletionThreshold: True
Microsoft.OptionalFeature.ExportDeletionThresholdValue: 500
Microsoft.OptionalFeature.FilterAAD: True
Microsoft.OptionalFeature.GroupFiltering: False
Microsoft.OptionalFeature.GroupWriteBack: False
Microsoft.OptionalFeature.HybridExchange: False
Microsoft.OptionalFeature.UserWriteBack: False
Microsoft.SynchronizationOption.AnchorAttribute: mS-DS-ConsistencyGuid
Microsoft.SynchronizationOption.CustomAttribute:
Microsoft.SynchronizationOption.JoinCriteria: AlwaysProvision
Microsoft.SynchronizationOption.UPNAttribute: userPrincipalName
Microsoft.Synchronize.MaintenanceEnabled: True
Microsoft.Synchronize.NextStartTime: Thu, 22 Oct 2020 15:37:03 GMT
Microsoft.Synchronize.RunHistoryPurgeInterval: 7.00:00:00
Microsoft.Synchronize.ServerConfigurationVersion: 1.5.45.0
Microsoft.Synchronize.SchedulerSuspended: False
Microsoft.Synchronize.StagingMode: False
Microsoft.Synchronize.SynchronizationPolicy: Delta
Microsoft.Synchronize.SynchronizationSchedule: True
Microsoft.Synchronize.TimeInterval: 00:30:00
Microsoft.SystemInformation.MachineRole: RoleMemberServer
Microsoft.UserSignIn.DesktopSsoEnabled: True
Microsoft.UserSignIn.SignOnMethod: PassThroughAuthentication
Microsoft.UserWriteBack.Container:
Microsoft.UserWriteBack.Forest:
Microsoft.Version.SynchronizationRuleImmutableTag: V1


Azure AD Connect Sync Scheduler - Settings


AllowedSyncCycleInterval: 00:30:00
CurrentlyEffectiveSyncCycleInterval: 00:30:00
CustomizedSyncCycleInterval:
NextSyncCyclePolicyType: Delta
NextSyncCycleStartTimeInUTC: 22.10.2020 15:37:03
PurgeRunHistoryInterval: 7.00:00:00
SyncCycleEnabled: True
MaintenanceEnabled: True
StagingModeEnabled: False
SchedulerSuspended: False
SyncCycleInProgress: False


On 10/22/2020 1:19 PM, Michal Bruncko via samba wrote:
> hello mj
>
> we are school - we are syncing approx 500 users and several groups.
>
> we use pass-through authentication because AAD authentication was not 
> working (that time we used samba 4.8.12, but since moving to 4.12 I 
> didnt tested this), we have (as recommended) one AAD connector plus 
> two authentication agents deployed in our environment.
>
> the thing what I dont understand is the amount of data which is 
> connector getting each day (~50GB). I have to check AAD connector 
> tshoot tool to understand whether there are some issues (like sync 
> will not get properly completed and therefore it is always retrying).
>
> but from AAD point of view all looks fine - in AAD portal I see the 
> sync is done every 30 minutes without errors.
>
> but this is definitely not correct behaviour regards to CPU and BW 
> utilization...
>
> regards
> michal
>
>
> On 10/21/2020 9:16 PM, mj via samba wrote:
>> Hi Michal,
>>
>> Seems we are doing similar things at the moment: getting samba to 
>> work with azure AD.
>>
>> We also see the high CPU usage on the DC that the Azure AD Connect 
>> server connected to. Between 70 - 100 percent in our case.
>>
>> We are not seeing any replication issues after azure AD Connect, and 
>> I have a script that automatically checks replication every few minutes.
>>
>> I was the one reporting the highwatermark back in 2017, but we're not 
>> getting those now. Between 2017 and now, we stopped testing azure, 
>> and I took it up again only last week.
>>
>> On the samba DC side, we observe that samba is very buzy talking to 
>> the Azure AD Connect machine, even though we filter sync using one 
>> group containing only three members, and none of them are actually 
>> changing. (this is just for testing now) How many users are you syncing?
>>
>> Are you using pass-through authentication, or syncing password 
>> hashes? And is what you chose working for you, authentication-wise?
>>
>> Regards,
>> MJ
>>
>> On 10/21/20 7:10 PM, Michal Bruncko via samba wrote:
>>> ups, seems pictures (attachments in general) are not accepted here,
>>> screen (graph) is available here: 
>>> https://i.postimg.cc/xCk6k038/image-2020-10-21-190940.png
>>>
>>> On 10/21/2020 6:00 PM, Michal Bruncko wrote:
>>>> hello
>>>>
>>>> our AD domain is hosted by two samba AD domain controllers version 
>>>> 4.12.6
>>>> - replication between controllers is fine, no problems.
>>>> - no schema errors.
>>>> - no database errors, all fine.
>>>> - no CPU utilizations
>>>> - wthout noticeable bandwidth utilization
>>>>
>>>> Recently we have deployed Azure AD connector on dedicated windows 
>>>> system (system is domain member server). since this deployment we 
>>>> are observing following issues on DCs:
>>>> - CPU utilization issue (one CPU core fully utilized)
>>>> - high BW utilization
>>>> - replication issue messages:
>>>> [2020/10/21 17:41:55.043563,  0] 
>>>> ../../source4/rpc_server/drsuapi/getncchanges.c:2910(dcesrv_drsuapi_DsGetNCChanges) 
>>>>
>>>>   ../../source4/rpc_server/drsuapi/getncchanges.c:2910: 
>>>> DsGetNCChanges 2nd replication on DN DC= older highwatermark 
>>>> (last_dn CN=userXYZ,OU=Users,DC=)
>>>>
>>>>
>>>> and this is happening only on one DC server in time - the one, to 
>>>> which this AD connector is connected for doing AD to AAD sync tasks.
>>>>
>>>> More details:
>>>>
>>>> CPU: mostly only one CPU core from all system-assigned cores is 
>>>> utilized at 100%:
>>>>
>>>>
>>>> BW utilization: you can see example here (peak starts once the 
>>>> Azure AD connector connects to particular DC server) (notice the 
>>>> "uploaded" data - 54GB - value from DC system):
>>>>
>>>>
>>>>
>>>> Replicaton errors: repeating messages (example above) every each 
>>>> 4-5 seconds. the "last_dn" is changing during time slowly: it is 
>>>> changed to another (user) object each several hours.
>>>>
>>>> no other issues observed.
>>>>
>>>> - If we deactivate this Azure connector, all issues stopped (but of 
>>>> course we are out of sync with AAD)
>>>> - if we reboot/stop DC1 services (serving for Azure connector), the 
>>>> Azure connector switch to DC2 and same story happen again 
>>>> (CPU/bandwidth/replication logs)
>>>>
>>>> I've found similar issue reported back in 2017: 
>>>> https://lists.samba.org/archive/samba/2017-October/211756.html 
>>>> ([Samba] samba getting stuck, highwatermark replication issue?)
>>>>
>>>> seems this issue is still in place now. no difference.
>>>>
>>>>
>>>> does anyone else have similar issues? does anyone else how to 
>>>> resolve them? either on Azure AD connector side (there are various 
>>>> confiuration option available) or (possibly) on samba side?
>>>>
>>>>
>>>> thank you
>>>> michal
>>>>
>>>
>>
>
>