[Samba] Proper sysvol replication solution...

Achim Gottinger achim at ag-web.biz
Fri Aug 22 12:51:57 MDT 2014 

Am 22.08.2014 20:31, schrieb Rowland Penny:
> On 22/08/14 18:40, Achim Gottinger wrote:
>> Am 22.08.2014 17:09, schrieb Rowland Penny:
>>> On 22/08/14 16:03, L.P.H. van Belle wrote:
>>>> when i look on my sysvol and subfolders
>>>> and
>>>> getfacl /home/samba/sysvol/internal.domain.tld/
>>>> the on both servers is the same.
>>>> and when i look from within windows both shares are exacly the same 
>>>> with rights.
>>>> share files and folders.
>>>>
>>>> maybe im lucky ;-)
>>> Lucky ?? that's nearly impossible, I bet you win everything you 
>>> enter ;-)
>>>
>>>>
>>>> no, i dont need these mappings imo since i dont have any linux/unix 
>>>> user to the sysvol share.
>>>> makes life lot easier..
>>> Yes but it doesn't just affect sysvol, it can affect any share.
>>>
>>> Rowland
>>>> Louis
>>>>
>>>>
>>>>
>>>>> -----Oorspronkelijk bericht-----
>>>>> Van: ryana at reachtechfp.com
>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>> Verzonden: vrijdag 22 augustus 2014 16:47
>>>>> Aan: samba at lists.samba.org
>>>>> Onderwerp: Re: [Samba] Proper sysvol replication solution...
>>>>>
>>>>> Domain users don't enter the equation here though. We're talking the
>>>>> "well-known" RID's like Rowland just mentioned. In other words, if my
>>>>> "Authenticated Users" is XID 30001 on DC1 but it is 30009 on DC2, I
>>>>> would not want to clone the UID/GID info from DC1 to DC2, only
>>>>> the files
>>>>> and directories. That where the "idmap.ldb" file comes into play.
>>>>> Apparently it is created with random XID's when Samba is
>>>>> configured and
>>>>> ran for the first time on each machine. Does your script copy those
>>>>> UID/GID numbers over? If so, I was under the impression that such a
>>>>> thing was bad.
>>>>>
>>>>> I am currently setting up two virtual machines (VirtualBox) to
>>>>> play with
>>>>> your script. I want to understand how it works and this way I am not
>>>>> worried about destroying a client's domain while I am toying around.
>>>>>
>>>>> On 08/22/2014 10:34 AM, L.P.H. van Belle wrote:
>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>> Van: ryana at reachtechfp.com
>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>> Verzonden: vrijdag 22 augustus 2014 15:30
>>>>>>> Aan: samba at lists.samba.org
>>>>>>> Onderwerp: Re: [Samba] Proper sysvol replication solution...
>>>>>>>
>>>>>>> I know it is, on purpose. Copying the ACLs from one DC to
>>>>>>> another is bad
>>>>>>> because for some reason each DC may have a different ID for
>>>>>>> the built-in
>>>>>>> accounts. In fact I believe it was Rowland who was saying something
>>>>>>> about copying the "idmap.tdb" file over if you sync the ACLs.
>>>>>>> My method
>>>>>>> only copies over the files and directories so no ACLs get
>>>>>>> screwy. Maybe
>>>>>>> Rowland can reiterate what he said on the other thread.
>>>>>>>
>>>>>> Yes, your right, it can be different if you dont keep de the
>>>>> default settings.
>>>>>> Which should be ok. I dont have any reason to change the
>>>>> acls on sysvol.
>>>>>> And i only use my ADDC as DC and not as file server so i
>>>>> dont have any idmap problems.
>>>>>> And i only use windows users to access these shares.
>>>>>> No linux users or what have access to these machines. only
>>>>> me as linux user and windows user.
>>>>>> and that are 2 different accounts. my linux user is only use
>>>>> to maintain the system.
>>>>>>> I actually did use rsync for a while and then realized I was
>>>>>>> messing up
>>>>>>> after several machines stopped being able to access the sysvol
>>>>>>> share on
>>>>>>> the second DC. When I ran "samba-tool ntacl sysvolcheck" it
>>>>>>> would crash.
>>>>>>> I ran a reset and all was good, until the next rsync. I
>>>>>>> finally figured
>>>>>>> it out and removed rsync and the rsync cron job.
>>>>>>>
>>>>>>> So tell me, why are you using rsync and unison? I am also trying to
>>>>>>> figure out how your script figures out which way to use rsync
>>>>>>> (DC1->DC2
>>>>>>> or DC2->DC1) based on what changes are on each DC. In other
>>>>>>> words, would
>>>>>>> your script work if a directory and file was created on DC1 AND a
>>>>>>> directory was removed on DC2? Would the new stuff get copied
>>>>>>> to DC2 and
>>>>>>> would the directory be removed on DC1? Forgive me, but I am
>>>>> not a bash
>>>>>>> scripting god. I know the basics and can get my work done, but
>>>>>>> I am not
>>>>>>> understanding how your script does two-way stuff via the
>>>>> rsync portion.
>>>>>> Well normal questions.. so
>>>>>> If a file is remove from sysvol on DC1 it removes it also from DC2.
>>>>>> If at the same time a folder or file was remove from DC2
>>>>> then it does the same with it on DC1.
>>>>>> just test it.. you see.
>>>>>> And best tip of the day.. make a backup of your sysvol
>>>>> before you start,
>>>>>> when you rest things and you forget to remove the unison
>>>>> hash files you can delete you sysvol..
>>>>>> we need rsync to create the directory structure with
>>>>> extended attributes,
>>>>>> the (now) unison setup copies only the extened attributes on files.
>>>>>>
>>>>>> cron does the sync every 5 min for me and you can also use
>>>>> inotify for that.
>>>>>> You can search for that one on the list.
>>>>>>
>>>>>> Hope answere this helps you.
>>>>>>
>>>>>>
>>>>>>> On 08/22/2014 02:21 AM, L.P.H. van Belle wrote:
>>>>>>>> ah you didnt see my script...
>>>>>>>>
>>>>>>>> https://secure.bazuin.nl/scripts/3-setup-sysvol-bidirectional.sh
>>>>>>>>
>>>>>>>> You way is missing the ACL.. you need rsync with unison.
>>>>>>>> its all in my script.
>>>>>>>>
>>>>>>>>
>>>>>>>> Greetz,
>>>>>>>>
>>>>>>>> Louis
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>>>> Van: ryana at reachtechfp.com
>>>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>>>> Verzonden: vrijdag 22 augustus 2014 0:14
>>>>>>>>> Aan: samba at lists.samba.org
>>>>>>>>> Onderwerp: [Samba] Proper sysvol replication solution...
>>>>>>>>>
>>>>>>>>> I see the Samba guide suggests using rsync to keep sysvols in
>>>>>>>>> sync, but
>>>>>>>>> this poses a problem with ID's and it is only one-way. I have 
>>>>>>>>> been
>>>>>>>>> hesitant to suggest anything because of the flak I have
>>>>>>> been getting,
>>>>>>>>> but I do believe I have a much better solution that transfers
>>>>>>>>> files via
>>>>>>>>> SSH, is bi-directional (no more only editing group policy on one
>>>>>>>>> server), and does NOT set UID/GID information. In other
>>>>> words, it is
>>>>>>>>> PERFECT for sysvol replication, and has been working on
>>>>>>> several of my
>>>>>>>>> domains for around a year and a half without a hitch. The
>>>>>>>>> solution I am
>>>>>>>>> proposing is to use unison, which also works on Windows and (I
>>>>>>>>> think) Mac.
>>>>>>>>>
>>>>>>>>> The way I have unison working on my systems is to install
>>>>>>>>> unison on all
>>>>>>>>> DC's, which is required. You also need an SSH server and
>>>>>>> client on all
>>>>>>>>> DC's, but I assume most of you do anyway. Once they're
>>>>>>>>> installed, it is
>>>>>>>>> as simple as the command below. This will synchronizes changes
>>>>>>>>> BOTH WAYS
>>>>>>>>> without touching your UID/GID setup. If you're paranoid, you 
>>>>>>>>> could
>>>>>>>>> always do a sysvolreset when done though.
>>>>>>>>>
>>>>>>>>> unison -batch "/path/to/sysvol"
>>>>>>> "ssh://dc02.domain.lan//path/to/sysvol"
>>>>>>>>> If you do this at a command-line, it will prompt you for
>>>>>>> your password
>>>>>>>>> on the remote machine. This would prevent a cron job, but
>>>>> I overcame
>>>>>>>>> that as well. You can create an SSH key that does not require
>>>>>>>>> a password
>>>>>>>>> for the systems to use. This means you can now create a
>>>>> cron job to
>>>>>>>>> handle the replication every fifteen minutes or so. You
>>>>>>> could also use
>>>>>>>>> something like "incrond" to monitor for changes in the sysvol
>>>>>>>>> and launch
>>>>>>>>> unison as well, but I don't personally modify the sysvol 
>>>>>>>>> often, so
>>>>>>>>> replication every fifteen minutes works for me.
>>>>>>>>>
>>>>>>>>> To create an SSH key to allow password-less replication via
>>>>>>> unison, do
>>>>>>>>> the following.
>>>>>>>>>
>>>>>>>>> ssh-keygen -t dsa
>>>>>>>>>
>>>>>>>>> When it prompts for a file to save the key in, it should be
>>>>>>> your home
>>>>>>>>> directory in a ".ssh" directory. I run as root, so this is
>>>>>>>>> "/root/.ssh/id_dsa" for me. It will then prompt for a
>>>>>>> password. Ignore
>>>>>>>>> this and just press enter. It will ask you to verify the
>>>>>>>>> password. Press
>>>>>>>>> enter again. If you enter a password here, it cannot run
>>>>>>>>> without user input!
>>>>>>>>>
>>>>>>>>> Next, you need to copy the key to your other domain
>>>>>>>>> controllers. You can
>>>>>>>>> do so as follows. Note that my example is run as root.
>>>>>>> Substitute your
>>>>>>>>> user's path if needed.
>>>>>>>>>
>>>>>>>>> ssh-copy-id -i /root/.ssh/id_dsa.pub root at dc02.domain.lan
>>>>>>>>>
>>>>>>>>> Once that is done, login to the domain controller you copied
>>>>>>>>> the key to
>>>>>>>>> (in the example, dc02) and check "/root/.ssh/authorized_keys"
>>>>>>>>> to verify
>>>>>>>>> that the key was added and nothing unexpected is there. You
>>>>>>>>> can do this
>>>>>>>>> with "cat /root/.ssh/authorized_keys". You should see a key on
>>>>>>>>> a single
>>>>>>>>> line followed by the hostname of your primary domain
>>>>>>> controller. If it
>>>>>>>>> is there, they may now connect via SSH without a password!
>>>>>>>>>
>>>>>>>>> You may now copy the key to any other domain controllers in
>>>>>>>>> your domain
>>>>>>>>> so they trust the primary DC as well. After that, all that
>>>>>>> is left is
>>>>>>>>> the synchronization. I urge you to run the first synchronization
>>>>>>>>> manually, like below.
>>>>>>>>>
>>>>>>>>> unison "/path/to/sysvol" "ssh://dc02.domain.lan//path/to/sysvol"
>>>>>>>>>
>>>>>>>>> Make sure everything looks good, synchronize it, then repeat
>>>>>>>>> for each DC
>>>>>>>>> on your domain. Once done, you can create cron jobs to sync
>>>>>>>>> each server,
>>>>>>>>> or use a script like mine below. This script is on my
>>>>> primary DC. I
>>>>>>>>> actually only have two DC's, but I added more as an example here.
>>>>>>>>>
>>>>>>>>> #!/bin/bash
>>>>>>>>> SERVERLIST="dc02.domain.lan dc03.domain.lan dc04.domain.lan"
>>>>>>>>> SVPATH="/path/to/sysvol"
>>>>>>>>>
>>>>>>>>> # Synchronize all of the domain controllers
>>>>>>>>> for sLoop in ${SERVERLIST}
>>>>>>>>> do
>>>>>>>>>      unison -batch "${SVPATH}" "ssh://${sLoop}/${SVPATH}"
>>>>>>>>> done
>>>>>>>>>
>>>>>>>>> exit 0
>>>>>>>>>
>>>>>>>>> Now set that script to run in a cron job and you're golden.
>>>>>>> You could
>>>>>>>>> also setup "incrond" on all of your DC's and have it call
>>>>>>>>> unison to sync
>>>>>>>>> the other DC's whenever a write happens in your sysvol,
>>>>> but I do not
>>>>>>>>> need such a thing and have not personally tried it,
>>>>> though I have a
>>>>>>>>> fellow IT lead who has and likes it. My crontab job entry is
>>>>>>>>> listed below.
>>>>>>>>>
>>>>>>>>> 15 * * * * /root/sysvolsync.sh &> /dev/null
>>>>>>>>>
>>>>>>>>> I hope this helps somebody and if you see something wrong,
>>>>>>>>> feel free to
>>>>>>>>> let me know.
>>>>>>>>> -- 
>>>>>>>>> To unsubscribe from this list go to the following URL and read 
>>>>>>>>> the
>>>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>>>
>>>>>>>>>
>>>>>>> -- 
>>>>>>> To unsubscribe from this list go to the following URL and read the
>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>
>>>>>>>
>>>>> -- 
>>>>> To unsubscribe from this list go to the following URL and read the
>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>
>>>>>
>>>
>> Rowland do you remeber Michael Adam's patch here from april? Seems it 
>> did not yet make it into an stable release.
>
> Yes, I vaguely remember it, but it did not go far enough (no 
> disrespect meant Michael, I couldn't have got that far),  I 'think' it 
> only dealt with 'BUILTIN' objects, but they are a small part of the 
> well known RID's .
>
> Rowland
Good point,

An nss_idmap module or extention to nss_winbind which resolves 
xidNumbers to objectSID's might do the trick. It is not neccesarry to 
translate the SID into an name to solve the mapping problem.

achim~

More information about the samba mailing list