[Samba] Proper sysvol replication solution...

Achim Gottinger achim at ag-web.biz
Fri Aug 22 13:57:30 MDT 2014 

Am 22.08.2014 21:16, schrieb Rowland Penny:
> On 22/08/14 19:51, Achim Gottinger wrote:
>> Am 22.08.2014 20:31, schrieb Rowland Penny:
>>> On 22/08/14 18:40, Achim Gottinger wrote:
>>>> Am 22.08.2014 17:09, schrieb Rowland Penny:
>>>>> On 22/08/14 16:03, L.P.H. van Belle wrote:
>>>>>> when i look on my sysvol and subfolders
>>>>>> and
>>>>>> getfacl /home/samba/sysvol/internal.domain.tld/
>>>>>> the on both servers is the same.
>>>>>> and when i look from within windows both shares are exacly the 
>>>>>> same with rights.
>>>>>> share files and folders.
>>>>>>
>>>>>> maybe im lucky ;-)
>>>>> Lucky ?? that's nearly impossible, I bet you win everything you 
>>>>> enter ;-)
>>>>>
>>>>>>
>>>>>> no, i dont need these mappings imo since i dont have any 
>>>>>> linux/unix user to the sysvol share.
>>>>>> makes life lot easier..
>>>>> Yes but it doesn't just affect sysvol, it can affect any share.
>>>>>
>>>>> Rowland
>>>>>> Louis
>>>>>>
>>>>>>
>>>>>>
>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>> Van: ryana at reachtechfp.com
>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>> Verzonden: vrijdag 22 augustus 2014 16:47
>>>>>>> Aan: samba at lists.samba.org
>>>>>>> Onderwerp: Re: [Samba] Proper sysvol replication solution...
>>>>>>>
>>>>>>> Domain users don't enter the equation here though. We're talking 
>>>>>>> the
>>>>>>> "well-known" RID's like Rowland just mentioned. In other words, 
>>>>>>> if my
>>>>>>> "Authenticated Users" is XID 30001 on DC1 but it is 30009 on DC2, I
>>>>>>> would not want to clone the UID/GID info from DC1 to DC2, only
>>>>>>> the files
>>>>>>> and directories. That where the "idmap.ldb" file comes into play.
>>>>>>> Apparently it is created with random XID's when Samba is
>>>>>>> configured and
>>>>>>> ran for the first time on each machine. Does your script copy those
>>>>>>> UID/GID numbers over? If so, I was under the impression that such a
>>>>>>> thing was bad.
>>>>>>>
>>>>>>> I am currently setting up two virtual machines (VirtualBox) to
>>>>>>> play with
>>>>>>> your script. I want to understand how it works and this way I am 
>>>>>>> not
>>>>>>> worried about destroying a client's domain while I am toying 
>>>>>>> around.
>>>>>>>
>>>>>>> On 08/22/2014 10:34 AM, L.P.H. van Belle wrote:
>>>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>>>> Van: ryana at reachtechfp.com
>>>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>>>> Verzonden: vrijdag 22 augustus 2014 15:30
>>>>>>>>> Aan: samba at lists.samba.org
>>>>>>>>> Onderwerp: Re: [Samba] Proper sysvol replication solution...
>>>>>>>>>
>>>>>>>>> I know it is, on purpose. Copying the ACLs from one DC to
>>>>>>>>> another is bad
>>>>>>>>> because for some reason each DC may have a different ID for
>>>>>>>>> the built-in
>>>>>>>>> accounts. In fact I believe it was Rowland who was saying 
>>>>>>>>> something
>>>>>>>>> about copying the "idmap.tdb" file over if you sync the ACLs.
>>>>>>>>> My method
>>>>>>>>> only copies over the files and directories so no ACLs get
>>>>>>>>> screwy. Maybe
>>>>>>>>> Rowland can reiterate what he said on the other thread.
>>>>>>>>>
>>>>>>>> Yes, your right, it can be different if you dont keep de the
>>>>>>> default settings.
>>>>>>>> Which should be ok. I dont have any reason to change the
>>>>>>> acls on sysvol.
>>>>>>>> And i only use my ADDC as DC and not as file server so i
>>>>>>> dont have any idmap problems.
>>>>>>>> And i only use windows users to access these shares.
>>>>>>>> No linux users or what have access to these machines. only
>>>>>>> me as linux user and windows user.
>>>>>>>> and that are 2 different accounts. my linux user is only use
>>>>>>> to maintain the system.
>>>>>>>>> I actually did use rsync for a while and then realized I was
>>>>>>>>> messing up
>>>>>>>>> after several machines stopped being able to access the sysvol
>>>>>>>>> share on
>>>>>>>>> the second DC. When I ran "samba-tool ntacl sysvolcheck" it
>>>>>>>>> would crash.
>>>>>>>>> I ran a reset and all was good, until the next rsync. I
>>>>>>>>> finally figured
>>>>>>>>> it out and removed rsync and the rsync cron job.
>>>>>>>>>
>>>>>>>>> So tell me, why are you using rsync and unison? I am also 
>>>>>>>>> trying to
>>>>>>>>> figure out how your script figures out which way to use rsync
>>>>>>>>> (DC1->DC2
>>>>>>>>> or DC2->DC1) based on what changes are on each DC. In other
>>>>>>>>> words, would
>>>>>>>>> your script work if a directory and file was created on DC1 AND a
>>>>>>>>> directory was removed on DC2? Would the new stuff get copied
>>>>>>>>> to DC2 and
>>>>>>>>> would the directory be removed on DC1? Forgive me, but I am
>>>>>>> not a bash
>>>>>>>>> scripting god. I know the basics and can get my work done, but
>>>>>>>>> I am not
>>>>>>>>> understanding how your script does two-way stuff via the
>>>>>>> rsync portion.
>>>>>>>> Well normal questions.. so
>>>>>>>> If a file is remove from sysvol on DC1 it removes it also from 
>>>>>>>> DC2.
>>>>>>>> If at the same time a folder or file was remove from DC2
>>>>>>> then it does the same with it on DC1.
>>>>>>>> just test it.. you see.
>>>>>>>> And best tip of the day.. make a backup of your sysvol
>>>>>>> before you start,
>>>>>>>> when you rest things and you forget to remove the unison
>>>>>>> hash files you can delete you sysvol..
>>>>>>>> we need rsync to create the directory structure with
>>>>>>> extended attributes,
>>>>>>>> the (now) unison setup copies only the extened attributes on 
>>>>>>>> files.
>>>>>>>>
>>>>>>>> cron does the sync every 5 min for me and you can also use
>>>>>>> inotify for that.
>>>>>>>> You can search for that one on the list.
>>>>>>>>
>>>>>>>> Hope answere this helps you.
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 08/22/2014 02:21 AM, L.P.H. van Belle wrote:
>>>>>>>>>> ah you didnt see my script...
>>>>>>>>>>
>>>>>>>>>> https://secure.bazuin.nl/scripts/3-setup-sysvol-bidirectional.sh
>>>>>>>>>>
>>>>>>>>>> You way is missing the ACL.. you need rsync with unison.
>>>>>>>>>> its all in my script.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Greetz,
>>>>>>>>>>
>>>>>>>>>> Louis
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>>>>>> Van: ryana at reachtechfp.com
>>>>>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>>>>>> Verzonden: vrijdag 22 augustus 2014 0:14
>>>>>>>>>>> Aan: samba at lists.samba.org
>>>>>>>>>>> Onderwerp: [Samba] Proper sysvol replication solution...
>>>>>>>>>>>
>>>>>>>>>>> I see the Samba guide suggests using rsync to keep sysvols in
>>>>>>>>>>> sync, but
>>>>>>>>>>> this poses a problem with ID's and it is only one-way. I 
>>>>>>>>>>> have been
>>>>>>>>>>> hesitant to suggest anything because of the flak I have
>>>>>>>>> been getting,
>>>>>>>>>>> but I do believe I have a much better solution that transfers
>>>>>>>>>>> files via
>>>>>>>>>>> SSH, is bi-directional (no more only editing group policy on 
>>>>>>>>>>> one
>>>>>>>>>>> server), and does NOT set UID/GID information. In other
>>>>>>> words, it is
>>>>>>>>>>> PERFECT for sysvol replication, and has been working on
>>>>>>>>> several of my
>>>>>>>>>>> domains for around a year and a half without a hitch. The
>>>>>>>>>>> solution I am
>>>>>>>>>>> proposing is to use unison, which also works on Windows and (I
>>>>>>>>>>> think) Mac.
>>>>>>>>>>>
>>>>>>>>>>> The way I have unison working on my systems is to install
>>>>>>>>>>> unison on all
>>>>>>>>>>> DC's, which is required. You also need an SSH server and
>>>>>>>>> client on all
>>>>>>>>>>> DC's, but I assume most of you do anyway. Once they're
>>>>>>>>>>> installed, it is
>>>>>>>>>>> as simple as the command below. This will synchronizes changes
>>>>>>>>>>> BOTH WAYS
>>>>>>>>>>> without touching your UID/GID setup. If you're paranoid, you 
>>>>>>>>>>> could
>>>>>>>>>>> always do a sysvolreset when done though.
>>>>>>>>>>>
>>>>>>>>>>> unison -batch "/path/to/sysvol"
>>>>>>>>> "ssh://dc02.domain.lan//path/to/sysvol"
>>>>>>>>>>> If you do this at a command-line, it will prompt you for
>>>>>>>>> your password
>>>>>>>>>>> on the remote machine. This would prevent a cron job, but
>>>>>>> I overcame
>>>>>>>>>>> that as well. You can create an SSH key that does not require
>>>>>>>>>>> a password
>>>>>>>>>>> for the systems to use. This means you can now create a
>>>>>>> cron job to
>>>>>>>>>>> handle the replication every fifteen minutes or so. You
>>>>>>>>> could also use
>>>>>>>>>>> something like "incrond" to monitor for changes in the sysvol
>>>>>>>>>>> and launch
>>>>>>>>>>> unison as well, but I don't personally modify the sysvol 
>>>>>>>>>>> often, so
>>>>>>>>>>> replication every fifteen minutes works for me.
>>>>>>>>>>>
>>>>>>>>>>> To create an SSH key to allow password-less replication via
>>>>>>>>> unison, do
>>>>>>>>>>> the following.
>>>>>>>>>>>
>>>>>>>>>>> ssh-keygen -t dsa
>>>>>>>>>>>
>>>>>>>>>>> When it prompts for a file to save the key in, it should be
>>>>>>>>> your home
>>>>>>>>>>> directory in a ".ssh" directory. I run as root, so this is
>>>>>>>>>>> "/root/.ssh/id_dsa" for me. It will then prompt for a
>>>>>>>>> password. Ignore
>>>>>>>>>>> this and just press enter. It will ask you to verify the
>>>>>>>>>>> password. Press
>>>>>>>>>>> enter again. If you enter a password here, it cannot run
>>>>>>>>>>> without user input!
>>>>>>>>>>>
>>>>>>>>>>> Next, you need to copy the key to your other domain
>>>>>>>>>>> controllers. You can
>>>>>>>>>>> do so as follows. Note that my example is run as root.
>>>>>>>>> Substitute your
>>>>>>>>>>> user's path if needed.
>>>>>>>>>>>
>>>>>>>>>>> ssh-copy-id -i /root/.ssh/id_dsa.pub root at dc02.domain.lan
>>>>>>>>>>>
>>>>>>>>>>> Once that is done, login to the domain controller you copied
>>>>>>>>>>> the key to
>>>>>>>>>>> (in the example, dc02) and check "/root/.ssh/authorized_keys"
>>>>>>>>>>> to verify
>>>>>>>>>>> that the key was added and nothing unexpected is there. You
>>>>>>>>>>> can do this
>>>>>>>>>>> with "cat /root/.ssh/authorized_keys". You should see a key on
>>>>>>>>>>> a single
>>>>>>>>>>> line followed by the hostname of your primary domain
>>>>>>>>> controller. If it
>>>>>>>>>>> is there, they may now connect via SSH without a password!
>>>>>>>>>>>
>>>>>>>>>>> You may now copy the key to any other domain controllers in
>>>>>>>>>>> your domain
>>>>>>>>>>> so they trust the primary DC as well. After that, all that
>>>>>>>>> is left is
>>>>>>>>>>> the synchronization. I urge you to run the first 
>>>>>>>>>>> synchronization
>>>>>>>>>>> manually, like below.
>>>>>>>>>>>
>>>>>>>>>>> unison "/path/to/sysvol" 
>>>>>>>>>>> "ssh://dc02.domain.lan//path/to/sysvol"
>>>>>>>>>>>
>>>>>>>>>>> Make sure everything looks good, synchronize it, then repeat
>>>>>>>>>>> for each DC
>>>>>>>>>>> on your domain. Once done, you can create cron jobs to sync
>>>>>>>>>>> each server,
>>>>>>>>>>> or use a script like mine below. This script is on my
>>>>>>> primary DC. I
>>>>>>>>>>> actually only have two DC's, but I added more as an example 
>>>>>>>>>>> here.
>>>>>>>>>>>
>>>>>>>>>>> #!/bin/bash
>>>>>>>>>>> SERVERLIST="dc02.domain.lan dc03.domain.lan dc04.domain.lan"
>>>>>>>>>>> SVPATH="/path/to/sysvol"
>>>>>>>>>>>
>>>>>>>>>>> # Synchronize all of the domain controllers
>>>>>>>>>>> for sLoop in ${SERVERLIST}
>>>>>>>>>>> do
>>>>>>>>>>>      unison -batch "${SVPATH}" "ssh://${sLoop}/${SVPATH}"
>>>>>>>>>>> done
>>>>>>>>>>>
>>>>>>>>>>> exit 0
>>>>>>>>>>>
>>>>>>>>>>> Now set that script to run in a cron job and you're golden.
>>>>>>>>> You could
>>>>>>>>>>> also setup "incrond" on all of your DC's and have it call
>>>>>>>>>>> unison to sync
>>>>>>>>>>> the other DC's whenever a write happens in your sysvol,
>>>>>>> but I do not
>>>>>>>>>>> need such a thing and have not personally tried it,
>>>>>>> though I have a
>>>>>>>>>>> fellow IT lead who has and likes it. My crontab job entry is
>>>>>>>>>>> listed below.
>>>>>>>>>>>
>>>>>>>>>>> 15 * * * * /root/sysvolsync.sh &> /dev/null
>>>>>>>>>>>
>>>>>>>>>>> I hope this helps somebody and if you see something wrong,
>>>>>>>>>>> feel free to
>>>>>>>>>>> let me know.
>>>>>>>>>>> -- 
>>>>>>>>>>> To unsubscribe from this list go to the following URL and 
>>>>>>>>>>> read the
>>>>>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> To unsubscribe from this list go to the following URL and read 
>>>>>>>>> the
>>>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>>>
>>>>>>>>>
>>>>>>> -- 
>>>>>>> To unsubscribe from this list go to the following URL and read the
>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>
>>>>>>>
>>>>>
>>>> Rowland do you remeber Michael Adam's patch here from april? Seems 
>>>> it did not yet make it into an stable release.
>>>
>>> Yes, I vaguely remember it, but it did not go far enough (no 
>>> disrespect meant Michael, I couldn't have got that far),  I 'think' 
>>> it only dealt with 'BUILTIN' objects, but they are a small part of 
>>> the well known RID's .
>>>
>>> Rowland
>> Good point,
>>
>> An nss_idmap module or extention to nss_winbind which resolves 
>> xidNumbers to objectSID's might do the trick. It is not neccesarry to 
>> translate the SID into an name to solve the mapping problem.
>>
>> achim~
>>
> You mean like the reverse of what ever happens to create the entries 
> in idmap.ldb ?
>
> Rowland
>
Yes,

For your example in the other mail "getent group" would return

S-1-5-32-544:*:3000000

on the dc and

S-1-5-32-544:*:2000

More information about the samba mailing list