[Samba] Proper sysvol replication solution...

Achim Gottinger achim at ag-web.biz
Fri Aug 22 15:12:41 MDT 2014 

Am 22.08.2014 22:58, schrieb Rowland Penny:
> On 22/08/14 21:46, Achim Gottinger wrote:
>> Am 22.08.2014 22:28, schrieb Rowland Penny:
>>> On 22/08/14 20:57, Achim Gottinger wrote:
>>>> Am 22.08.2014 21:16, schrieb Rowland Penny:
>>>>> On 22/08/14 19:51, Achim Gottinger wrote:
>>>>>> Am 22.08.2014 20:31, schrieb Rowland Penny:
>>>>>>> On 22/08/14 18:40, Achim Gottinger wrote:
>>>>>>>> Am 22.08.2014 17:09, schrieb Rowland Penny:
>>>>>>>>> On 22/08/14 16:03, L.P.H. van Belle wrote:
>>>>>>>>>> when i look on my sysvol and subfolders
>>>>>>>>>> and
>>>>>>>>>> getfacl /home/samba/sysvol/internal.domain.tld/
>>>>>>>>>> the on both servers is the same.
>>>>>>>>>> and when i look from within windows both shares are exacly 
>>>>>>>>>> the same with rights.
>>>>>>>>>> share files and folders.
>>>>>>>>>>
>>>>>>>>>> maybe im lucky ;-)
>>>>>>>>> Lucky ?? that's nearly impossible, I bet you win everything 
>>>>>>>>> you enter ;-)
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> no, i dont need these mappings imo since i dont have any 
>>>>>>>>>> linux/unix user to the sysvol share.
>>>>>>>>>> makes life lot easier..
>>>>>>>>> Yes but it doesn't just affect sysvol, it can affect any share.
>>>>>>>>>
>>>>>>>>> Rowland
>>>>>>>>>> Louis
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>>>>>> Van: ryana at reachtechfp.com
>>>>>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>>>>>> Verzonden: vrijdag 22 augustus 2014 16:47
>>>>>>>>>>> Aan: samba at lists.samba.org
>>>>>>>>>>> Onderwerp: Re: [Samba] Proper sysvol replication solution...
>>>>>>>>>>>
>>>>>>>>>>> Domain users don't enter the equation here though. We're 
>>>>>>>>>>> talking the
>>>>>>>>>>> "well-known" RID's like Rowland just mentioned. In other 
>>>>>>>>>>> words, if my
>>>>>>>>>>> "Authenticated Users" is XID 30001 on DC1 but it is 30009 on 
>>>>>>>>>>> DC2, I
>>>>>>>>>>> would not want to clone the UID/GID info from DC1 to DC2, only
>>>>>>>>>>> the files
>>>>>>>>>>> and directories. That where the "idmap.ldb" file comes into 
>>>>>>>>>>> play.
>>>>>>>>>>> Apparently it is created with random XID's when Samba is
>>>>>>>>>>> configured and
>>>>>>>>>>> ran for the first time on each machine. Does your script 
>>>>>>>>>>> copy those
>>>>>>>>>>> UID/GID numbers over? If so, I was under the impression that 
>>>>>>>>>>> such a
>>>>>>>>>>> thing was bad.
>>>>>>>>>>>
>>>>>>>>>>> I am currently setting up two virtual machines (VirtualBox) to
>>>>>>>>>>> play with
>>>>>>>>>>> your script. I want to understand how it works and this way 
>>>>>>>>>>> I am not
>>>>>>>>>>> worried about destroying a client's domain while I am toying 
>>>>>>>>>>> around.
>>>>>>>>>>>
>>>>>>>>>>> On 08/22/2014 10:34 AM, L.P.H. van Belle wrote:
>>>>>>>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>>>>>>>> Van: ryana at reachtechfp.com
>>>>>>>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>>>>>>>> Verzonden: vrijdag 22 augustus 2014 15:30
>>>>>>>>>>>>> Aan: samba at lists.samba.org
>>>>>>>>>>>>> Onderwerp: Re: [Samba] Proper sysvol replication solution...
>>>>>>>>>>>>>
>>>>>>>>>>>>> I know it is, on purpose. Copying the ACLs from one DC to
>>>>>>>>>>>>> another is bad
>>>>>>>>>>>>> because for some reason each DC may have a different ID for
>>>>>>>>>>>>> the built-in
>>>>>>>>>>>>> accounts. In fact I believe it was Rowland who was saying 
>>>>>>>>>>>>> something
>>>>>>>>>>>>> about copying the "idmap.tdb" file over if you sync the ACLs.
>>>>>>>>>>>>> My method
>>>>>>>>>>>>> only copies over the files and directories so no ACLs get
>>>>>>>>>>>>> screwy. Maybe
>>>>>>>>>>>>> Rowland can reiterate what he said on the other thread.
>>>>>>>>>>>>>
>>>>>>>>>>>> Yes, your right, it can be different if you dont keep de the
>>>>>>>>>>> default settings.
>>>>>>>>>>>> Which should be ok. I dont have any reason to change the
>>>>>>>>>>> acls on sysvol.
>>>>>>>>>>>> And i only use my ADDC as DC and not as file server so i
>>>>>>>>>>> dont have any idmap problems.
>>>>>>>>>>>> And i only use windows users to access these shares.
>>>>>>>>>>>> No linux users or what have access to these machines. only
>>>>>>>>>>> me as linux user and windows user.
>>>>>>>>>>>> and that are 2 different accounts. my linux user is only use
>>>>>>>>>>> to maintain the system.
>>>>>>>>>>>>> I actually did use rsync for a while and then realized I was
>>>>>>>>>>>>> messing up
>>>>>>>>>>>>> after several machines stopped being able to access the 
>>>>>>>>>>>>> sysvol
>>>>>>>>>>>>> share on
>>>>>>>>>>>>> the second DC. When I ran "samba-tool ntacl sysvolcheck" it
>>>>>>>>>>>>> would crash.
>>>>>>>>>>>>> I ran a reset and all was good, until the next rsync. I
>>>>>>>>>>>>> finally figured
>>>>>>>>>>>>> it out and removed rsync and the rsync cron job.
>>>>>>>>>>>>>
>>>>>>>>>>>>> So tell me, why are you using rsync and unison? I am also 
>>>>>>>>>>>>> trying to
>>>>>>>>>>>>> figure out how your script figures out which way to use rsync
>>>>>>>>>>>>> (DC1->DC2
>>>>>>>>>>>>> or DC2->DC1) based on what changes are on each DC. In other
>>>>>>>>>>>>> words, would
>>>>>>>>>>>>> your script work if a directory and file was created on 
>>>>>>>>>>>>> DC1 AND a
>>>>>>>>>>>>> directory was removed on DC2? Would the new stuff get copied
>>>>>>>>>>>>> to DC2 and
>>>>>>>>>>>>> would the directory be removed on DC1? Forgive me, but I am
>>>>>>>>>>> not a bash
>>>>>>>>>>>>> scripting god. I know the basics and can get my work done, 
>>>>>>>>>>>>> but
>>>>>>>>>>>>> I am not
>>>>>>>>>>>>> understanding how your script does two-way stuff via the
>>>>>>>>>>> rsync portion.
>>>>>>>>>>>> Well normal questions.. so
>>>>>>>>>>>> If a file is remove from sysvol on DC1 it removes it also 
>>>>>>>>>>>> from DC2.
>>>>>>>>>>>> If at the same time a folder or file was remove from DC2
>>>>>>>>>>> then it does the same with it on DC1.
>>>>>>>>>>>> just test it.. you see.
>>>>>>>>>>>> And best tip of the day.. make a backup of your sysvol
>>>>>>>>>>> before you start,
>>>>>>>>>>>> when you rest things and you forget to remove the unison
>>>>>>>>>>> hash files you can delete you sysvol..
>>>>>>>>>>>> we need rsync to create the directory structure with
>>>>>>>>>>> extended attributes,
>>>>>>>>>>>> the (now) unison setup copies only the extened attributes 
>>>>>>>>>>>> on files.
>>>>>>>>>>>>
>>>>>>>>>>>> cron does the sync every 5 min for me and you can also use
>>>>>>>>>>> inotify for that.
>>>>>>>>>>>> You can search for that one on the list.
>>>>>>>>>>>>
>>>>>>>>>>>> Hope answere this helps you.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On 08/22/2014 02:21 AM, L.P.H. van Belle wrote:
>>>>>>>>>>>>>> ah you didnt see my script...
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://secure.bazuin.nl/scripts/3-setup-sysvol-bidirectional.sh 
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You way is missing the ACL.. you need rsync with unison.
>>>>>>>>>>>>>> its all in my script.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Greetz,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Louis
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>>>>>>>>>> Van: ryana at reachtechfp.com
>>>>>>>>>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>>>>>>>>>> Verzonden: vrijdag 22 augustus 2014 0:14
>>>>>>>>>>>>>>> Aan: samba at lists.samba.org
>>>>>>>>>>>>>>> Onderwerp: [Samba] Proper sysvol replication solution...
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I see the Samba guide suggests using rsync to keep 
>>>>>>>>>>>>>>> sysvols in
>>>>>>>>>>>>>>> sync, but
>>>>>>>>>>>>>>> this poses a problem with ID's and it is only one-way. I 
>>>>>>>>>>>>>>> have been
>>>>>>>>>>>>>>> hesitant to suggest anything because of the flak I have
>>>>>>>>>>>>> been getting,
>>>>>>>>>>>>>>> but I do believe I have a much better solution that 
>>>>>>>>>>>>>>> transfers
>>>>>>>>>>>>>>> files via
>>>>>>>>>>>>>>> SSH, is bi-directional (no more only editing group 
>>>>>>>>>>>>>>> policy on one
>>>>>>>>>>>>>>> server), and does NOT set UID/GID information. In other
>>>>>>>>>>> words, it is
>>>>>>>>>>>>>>> PERFECT for sysvol replication, and has been working on
>>>>>>>>>>>>> several of my
>>>>>>>>>>>>>>> domains for around a year and a half without a hitch. The
>>>>>>>>>>>>>>> solution I am
>>>>>>>>>>>>>>> proposing is to use unison, which also works on Windows 
>>>>>>>>>>>>>>> and (I
>>>>>>>>>>>>>>> think) Mac.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The way I have unison working on my systems is to install
>>>>>>>>>>>>>>> unison on all
>>>>>>>>>>>>>>> DC's, which is required. You also need an SSH server and
>>>>>>>>>>>>> client on all
>>>>>>>>>>>>>>> DC's, but I assume most of you do anyway. Once they're
>>>>>>>>>>>>>>> installed, it is
>>>>>>>>>>>>>>> as simple as the command below. This will synchronizes 
>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>> BOTH WAYS
>>>>>>>>>>>>>>> without touching your UID/GID setup. If you're paranoid, 
>>>>>>>>>>>>>>> you could
>>>>>>>>>>>>>>> always do a sysvolreset when done though.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> unison -batch "/path/to/sysvol"
>>>>>>>>>>>>> "ssh://dc02.domain.lan//path/to/sysvol"
>>>>>>>>>>>>>>> If you do this at a command-line, it will prompt you for
>>>>>>>>>>>>> your password
>>>>>>>>>>>>>>> on the remote machine. This would prevent a cron job, but
>>>>>>>>>>> I overcame
>>>>>>>>>>>>>>> that as well. You can create an SSH key that does not 
>>>>>>>>>>>>>>> require
>>>>>>>>>>>>>>> a password
>>>>>>>>>>>>>>> for the systems to use. This means you can now create a
>>>>>>>>>>> cron job to
>>>>>>>>>>>>>>> handle the replication every fifteen minutes or so. You
>>>>>>>>>>>>> could also use
>>>>>>>>>>>>>>> something like "incrond" to monitor for changes in the 
>>>>>>>>>>>>>>> sysvol
>>>>>>>>>>>>>>> and launch
>>>>>>>>>>>>>>> unison as well, but I don't personally modify the sysvol 
>>>>>>>>>>>>>>> often, so
>>>>>>>>>>>>>>> replication every fifteen minutes works for me.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To create an SSH key to allow password-less replication via
>>>>>>>>>>>>> unison, do
>>>>>>>>>>>>>>> the following.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ssh-keygen -t dsa
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> When it prompts for a file to save the key in, it should be
>>>>>>>>>>>>> your home
>>>>>>>>>>>>>>> directory in a ".ssh" directory. I run as root, so this is
>>>>>>>>>>>>>>> "/root/.ssh/id_dsa" for me. It will then prompt for a
>>>>>>>>>>>>> password. Ignore
>>>>>>>>>>>>>>> this and just press enter. It will ask you to verify the
>>>>>>>>>>>>>>> password. Press
>>>>>>>>>>>>>>> enter again. If you enter a password here, it cannot run
>>>>>>>>>>>>>>> without user input!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Next, you need to copy the key to your other domain
>>>>>>>>>>>>>>> controllers. You can
>>>>>>>>>>>>>>> do so as follows. Note that my example is run as root.
>>>>>>>>>>>>> Substitute your
>>>>>>>>>>>>>>> user's path if needed.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ssh-copy-id -i /root/.ssh/id_dsa.pub root at dc02.domain.lan
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Once that is done, login to the domain controller you 
>>>>>>>>>>>>>>> copied
>>>>>>>>>>>>>>> the key to
>>>>>>>>>>>>>>> (in the example, dc02) and check 
>>>>>>>>>>>>>>> "/root/.ssh/authorized_keys"
>>>>>>>>>>>>>>> to verify
>>>>>>>>>>>>>>> that the key was added and nothing unexpected is there. You
>>>>>>>>>>>>>>> can do this
>>>>>>>>>>>>>>> with "cat /root/.ssh/authorized_keys". You should see a 
>>>>>>>>>>>>>>> key on
>>>>>>>>>>>>>>> a single
>>>>>>>>>>>>>>> line followed by the hostname of your primary domain
>>>>>>>>>>>>> controller. If it
>>>>>>>>>>>>>>> is there, they may now connect via SSH without a password!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You may now copy the key to any other domain controllers in
>>>>>>>>>>>>>>> your domain
>>>>>>>>>>>>>>> so they trust the primary DC as well. After that, all that
>>>>>>>>>>>>> is left is
>>>>>>>>>>>>>>> the synchronization. I urge you to run the first 
>>>>>>>>>>>>>>> synchronization
>>>>>>>>>>>>>>> manually, like below.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> unison "/path/to/sysvol" 
>>>>>>>>>>>>>>> "ssh://dc02.domain.lan//path/to/sysvol"
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Make sure everything looks good, synchronize it, then 
>>>>>>>>>>>>>>> repeat
>>>>>>>>>>>>>>> for each DC
>>>>>>>>>>>>>>> on your domain. Once done, you can create cron jobs to sync
>>>>>>>>>>>>>>> each server,
>>>>>>>>>>>>>>> or use a script like mine below. This script is on my
>>>>>>>>>>> primary DC. I
>>>>>>>>>>>>>>> actually only have two DC's, but I added more as an 
>>>>>>>>>>>>>>> example here.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> #!/bin/bash
>>>>>>>>>>>>>>> SERVERLIST="dc02.domain.lan dc03.domain.lan 
>>>>>>>>>>>>>>> dc04.domain.lan"
>>>>>>>>>>>>>>> SVPATH="/path/to/sysvol"
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> # Synchronize all of the domain controllers
>>>>>>>>>>>>>>> for sLoop in ${SERVERLIST}
>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>      unison -batch "${SVPATH}" "ssh://${sLoop}/${SVPATH}"
>>>>>>>>>>>>>>> done
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> exit 0
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Now set that script to run in a cron job and you're golden.
>>>>>>>>>>>>> You could
>>>>>>>>>>>>>>> also setup "incrond" on all of your DC's and have it call
>>>>>>>>>>>>>>> unison to sync
>>>>>>>>>>>>>>> the other DC's whenever a write happens in your sysvol,
>>>>>>>>>>> but I do not
>>>>>>>>>>>>>>> need such a thing and have not personally tried it,
>>>>>>>>>>> though I have a
>>>>>>>>>>>>>>> fellow IT lead who has and likes it. My crontab job 
>>>>>>>>>>>>>>> entry is
>>>>>>>>>>>>>>> listed below.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 15 * * * * /root/sysvolsync.sh &> /dev/null
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I hope this helps somebody and if you see something wrong,
>>>>>>>>>>>>>>> feel free to
>>>>>>>>>>>>>>> let me know.
>>>>>>>>>>>>>>> -- 
>>>>>>>>>>>>>>> To unsubscribe from this list go to the following URL 
>>>>>>>>>>>>>>> and read the
>>>>>>>>>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>> -- 
>>>>>>>>>>>>> To unsubscribe from this list go to the following URL and 
>>>>>>>>>>>>> read the
>>>>>>>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>> -- 
>>>>>>>>>>> To unsubscribe from this list go to the following URL and 
>>>>>>>>>>> read the
>>>>>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>> Rowland do you remeber Michael Adam's patch here from april? 
>>>>>>>> Seems it did not yet make it into an stable release.
>>>>>>>
>>>>>>> Yes, I vaguely remember it, but it did not go far enough (no 
>>>>>>> disrespect meant Michael, I couldn't have got that far),  I 
>>>>>>> 'think' it only dealt with 'BUILTIN' objects, but they are a 
>>>>>>> small part of the well known RID's .
>>>>>>>
>>>>>>> Rowland
>>>>>> Good point,
>>>>>>
>>>>>> An nss_idmap module or extention to nss_winbind which resolves 
>>>>>> xidNumbers to objectSID's might do the trick. It is not 
>>>>>> neccesarry to translate the SID into an name to solve the mapping 
>>>>>> problem.
>>>>>>
>>>>>> achim~
>>>>>>
>>>>> You mean like the reverse of what ever happens to create the 
>>>>> entries in idmap.ldb ?
>>>>>
>>>>> Rowland
>>>>>
>>>> Yes,
>>>>
>>>> For your example in the other mail "getent group" would return
>>>>
>>>> S-1-5-32-544:*:3000000
>>>>
>>>> on the dc and
>>>>
>>>> S-1-5-32-544:*:2000
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>> That would be good, but it would be better if was:
>>>
>>> administrators:*:3000000
>>>
>>> on the dc and
>>>
>>> administrators:*:2000
>>>
>>> Rowland
>>>
>>
>> Odd thing is the sid to name mapping is already there 
>> (librpc/idl/security.idl) for well know sid's and rid's. Makes it 
>> even more strange that only BUILDIN groups had been implemented by 
>> the patch.
>>
>> @steve you would force the users to reserve an predefined range of 
>> numbers in their usermanagement for windows standard groups und users.
> And your problem with this is ???
>
> If people could get their heads around the fact that they do not need 
> any local unix users and just have domain users instead, this would 
> not really be a problem.
>
> Rowland
>
Not my problem but one. Well-know-rid's for two different domains. The 
sid's differ in the domain part but would get the same fixed gid because 
they have an identical well-known rid part.

More information about the samba mailing list