[Samba] Proper sysvol replication solution...

Achim Gottinger achim at ag-web.biz
Fri Aug 22 11:40:58 MDT 2014 

Am 22.08.2014 17:09, schrieb Rowland Penny:
> On 22/08/14 16:03, L.P.H. van Belle wrote:
>> when i look on my sysvol and subfolders
>> and
>> getfacl /home/samba/sysvol/internal.domain.tld/
>> the on both servers is the same.
>> and when i look from within windows both shares are exacly the same 
>> with rights.
>> share files and folders.
>>
>> maybe im lucky ;-)
> Lucky ?? that's nearly impossible, I bet you win everything you enter ;-)
>
>>
>> no, i dont need these mappings imo since i dont have any linux/unix 
>> user to the sysvol share.
>> makes life lot easier..
> Yes but it doesn't just affect sysvol, it can affect any share.
>
> Rowland
>> Louis
>>
>>
>>
>>> -----Oorspronkelijk bericht-----
>>> Van: ryana at reachtechfp.com
>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>> Verzonden: vrijdag 22 augustus 2014 16:47
>>> Aan: samba at lists.samba.org
>>> Onderwerp: Re: [Samba] Proper sysvol replication solution...
>>>
>>> Domain users don't enter the equation here though. We're talking the
>>> "well-known" RID's like Rowland just mentioned. In other words, if my
>>> "Authenticated Users" is XID 30001 on DC1 but it is 30009 on DC2, I
>>> would not want to clone the UID/GID info from DC1 to DC2, only
>>> the files
>>> and directories. That where the "idmap.ldb" file comes into play.
>>> Apparently it is created with random XID's when Samba is
>>> configured and
>>> ran for the first time on each machine. Does your script copy those
>>> UID/GID numbers over? If so, I was under the impression that such a
>>> thing was bad.
>>>
>>> I am currently setting up two virtual machines (VirtualBox) to
>>> play with
>>> your script. I want to understand how it works and this way I am not
>>> worried about destroying a client's domain while I am toying around.
>>>
>>> On 08/22/2014 10:34 AM, L.P.H. van Belle wrote:
>>>>> -----Oorspronkelijk bericht-----
>>>>> Van: ryana at reachtechfp.com
>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>> Verzonden: vrijdag 22 augustus 2014 15:30
>>>>> Aan: samba at lists.samba.org
>>>>> Onderwerp: Re: [Samba] Proper sysvol replication solution...
>>>>>
>>>>> I know it is, on purpose. Copying the ACLs from one DC to
>>>>> another is bad
>>>>> because for some reason each DC may have a different ID for
>>>>> the built-in
>>>>> accounts. In fact I believe it was Rowland who was saying something
>>>>> about copying the "idmap.tdb" file over if you sync the ACLs.
>>>>> My method
>>>>> only copies over the files and directories so no ACLs get
>>>>> screwy. Maybe
>>>>> Rowland can reiterate what he said on the other thread.
>>>>>
>>>> Yes, your right, it can be different if you dont keep de the
>>> default settings.
>>>> Which should be ok. I dont have any reason to change the
>>> acls on sysvol.
>>>> And i only use my ADDC as DC and not as file server so i
>>> dont have any idmap problems.
>>>> And i only use windows users to access these shares.
>>>> No linux users or what have access to these machines. only
>>> me as linux user and windows user.
>>>> and that are 2 different accounts. my linux user is only use
>>> to maintain the system.
>>>>> I actually did use rsync for a while and then realized I was
>>>>> messing up
>>>>> after several machines stopped being able to access the sysvol
>>>>> share on
>>>>> the second DC. When I ran "samba-tool ntacl sysvolcheck" it
>>>>> would crash.
>>>>> I ran a reset and all was good, until the next rsync. I
>>>>> finally figured
>>>>> it out and removed rsync and the rsync cron job.
>>>>>
>>>>> So tell me, why are you using rsync and unison? I am also trying to
>>>>> figure out how your script figures out which way to use rsync
>>>>> (DC1->DC2
>>>>> or DC2->DC1) based on what changes are on each DC. In other
>>>>> words, would
>>>>> your script work if a directory and file was created on DC1 AND a
>>>>> directory was removed on DC2? Would the new stuff get copied
>>>>> to DC2 and
>>>>> would the directory be removed on DC1? Forgive me, but I am
>>> not a bash
>>>>> scripting god. I know the basics and can get my work done, but
>>>>> I am not
>>>>> understanding how your script does two-way stuff via the
>>> rsync portion.
>>>> Well normal questions.. so
>>>> If a file is remove from sysvol on DC1 it removes it also from DC2.
>>>> If at the same time a folder or file was remove from DC2
>>> then it does the same with it on DC1.
>>>> just test it.. you see.
>>>> And best tip of the day.. make a backup of your sysvol
>>> before you start,
>>>> when you rest things and you forget to remove the unison
>>> hash files you can delete you sysvol..
>>>> we need rsync to create the directory structure with
>>> extended attributes,
>>>> the (now) unison setup copies only the extened attributes on files.
>>>>
>>>> cron does the sync every 5 min for me and you can also use
>>> inotify for that.
>>>> You can search for that one on the list.
>>>>
>>>> Hope answere this helps you.
>>>>
>>>>
>>>>> On 08/22/2014 02:21 AM, L.P.H. van Belle wrote:
>>>>>> ah you didnt see my script...
>>>>>>
>>>>>> https://secure.bazuin.nl/scripts/3-setup-sysvol-bidirectional.sh
>>>>>>
>>>>>> You way is missing the ACL.. you need rsync with unison.
>>>>>> its all in my script.
>>>>>>
>>>>>>
>>>>>> Greetz,
>>>>>>
>>>>>> Louis
>>>>>>
>>>>>>
>>>>>>
>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>> Van: ryana at reachtechfp.com
>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>> Verzonden: vrijdag 22 augustus 2014 0:14
>>>>>>> Aan: samba at lists.samba.org
>>>>>>> Onderwerp: [Samba] Proper sysvol replication solution...
>>>>>>>
>>>>>>> I see the Samba guide suggests using rsync to keep sysvols in
>>>>>>> sync, but
>>>>>>> this poses a problem with ID's and it is only one-way. I have been
>>>>>>> hesitant to suggest anything because of the flak I have
>>>>> been getting,
>>>>>>> but I do believe I have a much better solution that transfers
>>>>>>> files via
>>>>>>> SSH, is bi-directional (no more only editing group policy on one
>>>>>>> server), and does NOT set UID/GID information. In other
>>> words, it is
>>>>>>> PERFECT for sysvol replication, and has been working on
>>>>> several of my
>>>>>>> domains for around a year and a half without a hitch. The
>>>>>>> solution I am
>>>>>>> proposing is to use unison, which also works on Windows and (I
>>>>>>> think) Mac.
>>>>>>>
>>>>>>> The way I have unison working on my systems is to install
>>>>>>> unison on all
>>>>>>> DC's, which is required. You also need an SSH server and
>>>>> client on all
>>>>>>> DC's, but I assume most of you do anyway. Once they're
>>>>>>> installed, it is
>>>>>>> as simple as the command below. This will synchronizes changes
>>>>>>> BOTH WAYS
>>>>>>> without touching your UID/GID setup. If you're paranoid, you could
>>>>>>> always do a sysvolreset when done though.
>>>>>>>
>>>>>>> unison -batch "/path/to/sysvol"
>>>>> "ssh://dc02.domain.lan//path/to/sysvol"
>>>>>>> If you do this at a command-line, it will prompt you for
>>>>> your password
>>>>>>> on the remote machine. This would prevent a cron job, but
>>> I overcame
>>>>>>> that as well. You can create an SSH key that does not require
>>>>>>> a password
>>>>>>> for the systems to use. This means you can now create a
>>> cron job to
>>>>>>> handle the replication every fifteen minutes or so. You
>>>>> could also use
>>>>>>> something like "incrond" to monitor for changes in the sysvol
>>>>>>> and launch
>>>>>>> unison as well, but I don't personally modify the sysvol often, so
>>>>>>> replication every fifteen minutes works for me.
>>>>>>>
>>>>>>> To create an SSH key to allow password-less replication via
>>>>> unison, do
>>>>>>> the following.
>>>>>>>
>>>>>>> ssh-keygen -t dsa
>>>>>>>
>>>>>>> When it prompts for a file to save the key in, it should be
>>>>> your home
>>>>>>> directory in a ".ssh" directory. I run as root, so this is
>>>>>>> "/root/.ssh/id_dsa" for me. It will then prompt for a
>>>>> password. Ignore
>>>>>>> this and just press enter. It will ask you to verify the
>>>>>>> password. Press
>>>>>>> enter again. If you enter a password here, it cannot run
>>>>>>> without user input!
>>>>>>>
>>>>>>> Next, you need to copy the key to your other domain
>>>>>>> controllers. You can
>>>>>>> do so as follows. Note that my example is run as root.
>>>>> Substitute your
>>>>>>> user's path if needed.
>>>>>>>
>>>>>>> ssh-copy-id -i /root/.ssh/id_dsa.pub root at dc02.domain.lan
>>>>>>>
>>>>>>> Once that is done, login to the domain controller you copied
>>>>>>> the key to
>>>>>>> (in the example, dc02) and check "/root/.ssh/authorized_keys"
>>>>>>> to verify
>>>>>>> that the key was added and nothing unexpected is there. You
>>>>>>> can do this
>>>>>>> with "cat /root/.ssh/authorized_keys". You should see a key on
>>>>>>> a single
>>>>>>> line followed by the hostname of your primary domain
>>>>> controller. If it
>>>>>>> is there, they may now connect via SSH without a password!
>>>>>>>
>>>>>>> You may now copy the key to any other domain controllers in
>>>>>>> your domain
>>>>>>> so they trust the primary DC as well. After that, all that
>>>>> is left is
>>>>>>> the synchronization. I urge you to run the first synchronization
>>>>>>> manually, like below.
>>>>>>>
>>>>>>> unison "/path/to/sysvol" "ssh://dc02.domain.lan//path/to/sysvol"
>>>>>>>
>>>>>>> Make sure everything looks good, synchronize it, then repeat
>>>>>>> for each DC
>>>>>>> on your domain. Once done, you can create cron jobs to sync
>>>>>>> each server,
>>>>>>> or use a script like mine below. This script is on my
>>> primary DC. I
>>>>>>> actually only have two DC's, but I added more as an example here.
>>>>>>>
>>>>>>> #!/bin/bash
>>>>>>> SERVERLIST="dc02.domain.lan dc03.domain.lan dc04.domain.lan"
>>>>>>> SVPATH="/path/to/sysvol"
>>>>>>>
>>>>>>> # Synchronize all of the domain controllers
>>>>>>> for sLoop in ${SERVERLIST}
>>>>>>> do
>>>>>>>      unison -batch "${SVPATH}" "ssh://${sLoop}/${SVPATH}"
>>>>>>> done
>>>>>>>
>>>>>>> exit 0
>>>>>>>
>>>>>>> Now set that script to run in a cron job and you're golden.
>>>>> You could
>>>>>>> also setup "incrond" on all of your DC's and have it call
>>>>>>> unison to sync
>>>>>>> the other DC's whenever a write happens in your sysvol,
>>> but I do not
>>>>>>> need such a thing and have not personally tried it,
>>> though I have a
>>>>>>> fellow IT lead who has and likes it. My crontab job entry is
>>>>>>> listed below.
>>>>>>>
>>>>>>> 15 * * * * /root/sysvolsync.sh &> /dev/null
>>>>>>>
>>>>>>> I hope this helps somebody and if you see something wrong,
>>>>>>> feel free to
>>>>>>> let me know.
>>>>>>> -- 
>>>>>>> To unsubscribe from this list go to the following URL and read the
>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>
>>>>>>>
>>>>> -- 
>>>>> To unsubscribe from this list go to the following URL and read the
>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>
>>>>>
>>> -- 
>>> To unsubscribe from this list go to the following URL and read the
>>> instructions:  https://lists.samba.org/mailman/options/samba
>>>
>>>
>
Rowland do you remeber Michael Adam's patch here from april? Seems it 
did not yet make it into an stable release.

> Hi!
>
> Attached is a patch that fixes --gid-info and hence "getent
> group" for builtins on the DC. Note that this will not produce
> the same GIDs as on a member.
>
> I need to do more testing with this but wanted to
> share it for those who are interested.
>
> (And also remember that you should not use  a range below 1000
> for id mapping on a member on modern linux/unix systems, because
> you risk clashes with system groups.)
>
> Cheers - Michael
>
> Note: cross-posting to samba-technical since this involves a patch...


> BTW, note that currently the getent group enumeration
> only prints domain groups not domain local groups (aliases)
> or builtins, but at least for builtins, the form
> "getent group BULTIN\\Administrators" works with the previous
> patch, so we could add that to enumeration as well.
> But frankly enumeration is not important. The individual
> calls are what is required to work.
>
> Cheers - Michael
>

More information about the samba mailing list