[Samba] Proper sysvol replication solution...

Fri Aug 22 12:31:22 MDT 2014

On 22/08/14 18:40, Achim Gottinger wrote:
> Am 22.08.2014 17:09, schrieb Rowland Penny:
>> On 22/08/14 16:03, L.P.H. van Belle wrote:
>>> when i look on my sysvol and subfolders
>>> and
>>> getfacl /home/samba/sysvol/internal.domain.tld/
>>> the on both servers is the same.
>>> and when i look from within windows both shares are exacly the same 
>>> with rights.
>>> share files and folders.
>>>
>>> maybe im lucky ;-)
>> Lucky ?? that's nearly impossible, I bet you win everything you enter 
>> ;-)
>>
>>>
>>> no, i dont need these mappings imo since i dont have any linux/unix 
>>> user to the sysvol share.
>>> makes life lot easier..
>> Yes but it doesn't just affect sysvol, it can affect any share.
>>
>> Rowland
>>> Louis
>>>
>>>
>>>
>>>> -----Oorspronkelijk bericht-----
>>>> Van: ryana at reachtechfp.com
>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>> Verzonden: vrijdag 22 augustus 2014 16:47
>>>> Aan: samba at lists.samba.org
>>>> Onderwerp: Re: [Samba] Proper sysvol replication solution...
>>>>
>>>> Domain users don't enter the equation here though. We're talking the
>>>> "well-known" RID's like Rowland just mentioned. In other words, if my
>>>> "Authenticated Users" is XID 30001 on DC1 but it is 30009 on DC2, I
>>>> would not want to clone the UID/GID info from DC1 to DC2, only
>>>> the files
>>>> and directories. That where the "idmap.ldb" file comes into play.
>>>> Apparently it is created with random XID's when Samba is
>>>> configured and
>>>> ran for the first time on each machine. Does your script copy those
>>>> UID/GID numbers over? If so, I was under the impression that such a
>>>> thing was bad.
>>>>
>>>> I am currently setting up two virtual machines (VirtualBox) to
>>>> play with
>>>> your script. I want to understand how it works and this way I am not
>>>> worried about destroying a client's domain while I am toying around.
>>>>
>>>> On 08/22/2014 10:34 AM, L.P.H. van Belle wrote:
>>>>>> -----Oorspronkelijk bericht-----
>>>>>> Van: ryana at reachtechfp.com
>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>> Verzonden: vrijdag 22 augustus 2014 15:30
>>>>>> Aan: samba at lists.samba.org
>>>>>> Onderwerp: Re: [Samba] Proper sysvol replication solution...
>>>>>>
>>>>>> I know it is, on purpose. Copying the ACLs from one DC to
>>>>>> another is bad
>>>>>> because for some reason each DC may have a different ID for
>>>>>> the built-in
>>>>>> accounts. In fact I believe it was Rowland who was saying something
>>>>>> about copying the "idmap.tdb" file over if you sync the ACLs.
>>>>>> My method
>>>>>> only copies over the files and directories so no ACLs get
>>>>>> screwy. Maybe
>>>>>> Rowland can reiterate what he said on the other thread.
>>>>>>
>>>>> Yes, your right, it can be different if you dont keep de the
>>>> default settings.
>>>>> Which should be ok. I dont have any reason to change the
>>>> acls on sysvol.
>>>>> And i only use my ADDC as DC and not as file server so i
>>>> dont have any idmap problems.
>>>>> And i only use windows users to access these shares.
>>>>> No linux users or what have access to these machines. only
>>>> me as linux user and windows user.
>>>>> and that are 2 different accounts. my linux user is only use
>>>> to maintain the system.
>>>>>> I actually did use rsync for a while and then realized I was
>>>>>> messing up
>>>>>> after several machines stopped being able to access the sysvol
>>>>>> share on
>>>>>> the second DC. When I ran "samba-tool ntacl sysvolcheck" it
>>>>>> would crash.
>>>>>> I ran a reset and all was good, until the next rsync. I
>>>>>> finally figured
>>>>>> it out and removed rsync and the rsync cron job.
>>>>>>
>>>>>> So tell me, why are you using rsync and unison? I am also trying to
>>>>>> figure out how your script figures out which way to use rsync
>>>>>> (DC1->DC2
>>>>>> or DC2->DC1) based on what changes are on each DC. In other
>>>>>> words, would
>>>>>> your script work if a directory and file was created on DC1 AND a
>>>>>> directory was removed on DC2? Would the new stuff get copied
>>>>>> to DC2 and
>>>>>> would the directory be removed on DC1? Forgive me, but I am
>>>> not a bash
>>>>>> scripting god. I know the basics and can get my work done, but
>>>>>> I am not
>>>>>> understanding how your script does two-way stuff via the
>>>> rsync portion.
>>>>> Well normal questions.. so
>>>>> If a file is remove from sysvol on DC1 it removes it also from DC2.
>>>>> If at the same time a folder or file was remove from DC2
>>>> then it does the same with it on DC1.
>>>>> just test it.. you see.
>>>>> And best tip of the day.. make a backup of your sysvol
>>>> before you start,
>>>>> when you rest things and you forget to remove the unison
>>>> hash files you can delete you sysvol..
>>>>> we need rsync to create the directory structure with
>>>> extended attributes,
>>>>> the (now) unison setup copies only the extened attributes on files.
>>>>>
>>>>> cron does the sync every 5 min for me and you can also use
>>>> inotify for that.
>>>>> You can search for that one on the list.
>>>>>
>>>>> Hope answere this helps you.
>>>>>
>>>>>
>>>>>> On 08/22/2014 02:21 AM, L.P.H. van Belle wrote:
>>>>>>> ah you didnt see my script...
>>>>>>>
>>>>>>> https://secure.bazuin.nl/scripts/3-setup-sysvol-bidirectional.sh
>>>>>>>
>>>>>>> You way is missing the ACL.. you need rsync with unison.
>>>>>>> its all in my script.
>>>>>>>
>>>>>>>
>>>>>>> Greetz,
>>>>>>>
>>>>>>> Louis
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>>> Van: ryana at reachtechfp.com
>>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>>> Verzonden: vrijdag 22 augustus 2014 0:14
>>>>>>>> Aan: samba at lists.samba.org
>>>>>>>> Onderwerp: [Samba] Proper sysvol replication solution...
>>>>>>>>
>>>>>>>> I see the Samba guide suggests using rsync to keep sysvols in
>>>>>>>> sync, but
>>>>>>>> this poses a problem with ID's and it is only one-way. I have been
>>>>>>>> hesitant to suggest anything because of the flak I have
>>>>>> been getting,
>>>>>>>> but I do believe I have a much better solution that transfers
>>>>>>>> files via
>>>>>>>> SSH, is bi-directional (no more only editing group policy on one
>>>>>>>> server), and does NOT set UID/GID information. In other
>>>> words, it is
>>>>>>>> PERFECT for sysvol replication, and has been working on
>>>>>> several of my
>>>>>>>> domains for around a year and a half without a hitch. The
>>>>>>>> solution I am
>>>>>>>> proposing is to use unison, which also works on Windows and (I
>>>>>>>> think) Mac.
>>>>>>>>
>>>>>>>> The way I have unison working on my systems is to install
>>>>>>>> unison on all
>>>>>>>> DC's, which is required. You also need an SSH server and
>>>>>> client on all
>>>>>>>> DC's, but I assume most of you do anyway. Once they're
>>>>>>>> installed, it is
>>>>>>>> as simple as the command below. This will synchronizes changes
>>>>>>>> BOTH WAYS
>>>>>>>> without touching your UID/GID setup. If you're paranoid, you could
>>>>>>>> always do a sysvolreset when done though.
>>>>>>>>
>>>>>>>> unison -batch "/path/to/sysvol"
>>>>>> "ssh://dc02.domain.lan//path/to/sysvol"
>>>>>>>> If you do this at a command-line, it will prompt you for
>>>>>> your password
>>>>>>>> on the remote machine. This would prevent a cron job, but
>>>> I overcame
>>>>>>>> that as well. You can create an SSH key that does not require
>>>>>>>> a password
>>>>>>>> for the systems to use. This means you can now create a
>>>> cron job to
>>>>>>>> handle the replication every fifteen minutes or so. You
>>>>>> could also use
>>>>>>>> something like "incrond" to monitor for changes in the sysvol
>>>>>>>> and launch
>>>>>>>> unison as well, but I don't personally modify the sysvol often, so
>>>>>>>> replication every fifteen minutes works for me.
>>>>>>>>
>>>>>>>> To create an SSH key to allow password-less replication via
>>>>>> unison, do
>>>>>>>> the following.
>>>>>>>>
>>>>>>>> ssh-keygen -t dsa
>>>>>>>>
>>>>>>>> When it prompts for a file to save the key in, it should be
>>>>>> your home
>>>>>>>> directory in a ".ssh" directory. I run as root, so this is
>>>>>>>> "/root/.ssh/id_dsa" for me. It will then prompt for a
>>>>>> password. Ignore
>>>>>>>> this and just press enter. It will ask you to verify the
>>>>>>>> password. Press
>>>>>>>> enter again. If you enter a password here, it cannot run
>>>>>>>> without user input!
>>>>>>>>
>>>>>>>> Next, you need to copy the key to your other domain
>>>>>>>> controllers. You can
>>>>>>>> do so as follows. Note that my example is run as root.
>>>>>> Substitute your
>>>>>>>> user's path if needed.
>>>>>>>>
>>>>>>>> ssh-copy-id -i /root/.ssh/id_dsa.pub root at dc02.domain.lan
>>>>>>>>
>>>>>>>> Once that is done, login to the domain controller you copied
>>>>>>>> the key to
>>>>>>>> (in the example, dc02) and check "/root/.ssh/authorized_keys"
>>>>>>>> to verify
>>>>>>>> that the key was added and nothing unexpected is there. You
>>>>>>>> can do this
>>>>>>>> with "cat /root/.ssh/authorized_keys". You should see a key on
>>>>>>>> a single
>>>>>>>> line followed by the hostname of your primary domain
>>>>>> controller. If it
>>>>>>>> is there, they may now connect via SSH without a password!
>>>>>>>>
>>>>>>>> You may now copy the key to any other domain controllers in
>>>>>>>> your domain
>>>>>>>> so they trust the primary DC as well. After that, all that
>>>>>> is left is
>>>>>>>> the synchronization. I urge you to run the first synchronization
>>>>>>>> manually, like below.
>>>>>>>>
>>>>>>>> unison "/path/to/sysvol" "ssh://dc02.domain.lan//path/to/sysvol"
>>>>>>>>
>>>>>>>> Make sure everything looks good, synchronize it, then repeat
>>>>>>>> for each DC
>>>>>>>> on your domain. Once done, you can create cron jobs to sync
>>>>>>>> each server,
>>>>>>>> or use a script like mine below. This script is on my
>>>> primary DC. I
>>>>>>>> actually only have two DC's, but I added more as an example here.
>>>>>>>>
>>>>>>>> #!/bin/bash
>>>>>>>> SERVERLIST="dc02.domain.lan dc03.domain.lan dc04.domain.lan"
>>>>>>>> SVPATH="/path/to/sysvol"
>>>>>>>>
>>>>>>>> # Synchronize all of the domain controllers
>>>>>>>> for sLoop in ${SERVERLIST}
>>>>>>>> do
>>>>>>>>      unison -batch "${SVPATH}" "ssh://${sLoop}/${SVPATH}"
>>>>>>>> done
>>>>>>>>
>>>>>>>> exit 0
>>>>>>>>
>>>>>>>> Now set that script to run in a cron job and you're golden.
>>>>>> You could
>>>>>>>> also setup "incrond" on all of your DC's and have it call
>>>>>>>> unison to sync
>>>>>>>> the other DC's whenever a write happens in your sysvol,
>>>> but I do not
>>>>>>>> need such a thing and have not personally tried it,
>>>> though I have a
>>>>>>>> fellow IT lead who has and likes it. My crontab job entry is
>>>>>>>> listed below.
>>>>>>>>
>>>>>>>> 15 * * * * /root/sysvolsync.sh &> /dev/null
>>>>>>>>
>>>>>>>> I hope this helps somebody and if you see something wrong,
>>>>>>>> feel free to
>>>>>>>> let me know.
>>>>>>>> -- 
>>>>>>>> To unsubscribe from this list go to the following URL and read the
>>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>>
>>>>>>>>
>>>>>> -- 
>>>>>> To unsubscribe from this list go to the following URL and read the
>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>
>>>>>>
>>>> -- 
>>>> To unsubscribe from this list go to the following URL and read the
>>>> instructions:  https://lists.samba.org/mailman/options/samba
>>>>
>>>>
>>
> Rowland do you remeber Michael Adam's patch here from april? Seems it 
> did not yet make it into an stable release.

Yes, I vaguely remember it, but it did not go far enough (no disrespect 
meant Michael, I couldn't have got that far),  I 'think' it only dealt 
with 'BUILTIN' objects, but they are a small part of the well known RID's .

Rowland

>
>> Hi!
>>
>> Attached is a patch that fixes --gid-info and hence "getent
>> group" for builtins on the DC. Note that this will not produce
>> the same GIDs as on a member.
>>
>> I need to do more testing with this but wanted to
>> share it for those who are interested.
>>
>> (And also remember that you should not use  a range below 1000
>> for id mapping on a member on modern linux/unix systems, because
>> you risk clashes with system groups.)
>>
>> Cheers - Michael
>>
>> Note: cross-posting to samba-technical since this involves a patch...
>
>
>> BTW, note that currently the getent group enumeration
>> only prints domain groups not domain local groups (aliases)
>> or builtins, but at least for builtins, the form
>> "getent group BULTIN\\Administrators" works with the previous
>> patch, so we could add that to enumeration as well.
>> But frankly enumeration is not important. The individual
>> calls are what is required to work.
>>
>> Cheers - Michael
>>
>
>
>