[Samba] Proper sysvol replication solution...

Fri Aug 22 14:28:03 MDT 2014

On 22/08/14 20:57, Achim Gottinger wrote:
> Am 22.08.2014 21:16, schrieb Rowland Penny:
>> On 22/08/14 19:51, Achim Gottinger wrote:
>>> Am 22.08.2014 20:31, schrieb Rowland Penny:
>>>> On 22/08/14 18:40, Achim Gottinger wrote:
>>>>> Am 22.08.2014 17:09, schrieb Rowland Penny:
>>>>>> On 22/08/14 16:03, L.P.H. van Belle wrote:
>>>>>>> when i look on my sysvol and subfolders
>>>>>>> and
>>>>>>> getfacl /home/samba/sysvol/internal.domain.tld/
>>>>>>> the on both servers is the same.
>>>>>>> and when i look from within windows both shares are exacly the 
>>>>>>> same with rights.
>>>>>>> share files and folders.
>>>>>>>
>>>>>>> maybe im lucky ;-)
>>>>>> Lucky ?? that's nearly impossible, I bet you win everything you 
>>>>>> enter ;-)
>>>>>>
>>>>>>>
>>>>>>> no, i dont need these mappings imo since i dont have any 
>>>>>>> linux/unix user to the sysvol share.
>>>>>>> makes life lot easier..
>>>>>> Yes but it doesn't just affect sysvol, it can affect any share.
>>>>>>
>>>>>> Rowland
>>>>>>> Louis
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>>> Van: ryana at reachtechfp.com
>>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>>> Verzonden: vrijdag 22 augustus 2014 16:47
>>>>>>>> Aan: samba at lists.samba.org
>>>>>>>> Onderwerp: Re: [Samba] Proper sysvol replication solution...
>>>>>>>>
>>>>>>>> Domain users don't enter the equation here though. We're 
>>>>>>>> talking the
>>>>>>>> "well-known" RID's like Rowland just mentioned. In other words, 
>>>>>>>> if my
>>>>>>>> "Authenticated Users" is XID 30001 on DC1 but it is 30009 on 
>>>>>>>> DC2, I
>>>>>>>> would not want to clone the UID/GID info from DC1 to DC2, only
>>>>>>>> the files
>>>>>>>> and directories. That where the "idmap.ldb" file comes into play.
>>>>>>>> Apparently it is created with random XID's when Samba is
>>>>>>>> configured and
>>>>>>>> ran for the first time on each machine. Does your script copy 
>>>>>>>> those
>>>>>>>> UID/GID numbers over? If so, I was under the impression that 
>>>>>>>> such a
>>>>>>>> thing was bad.
>>>>>>>>
>>>>>>>> I am currently setting up two virtual machines (VirtualBox) to
>>>>>>>> play with
>>>>>>>> your script. I want to understand how it works and this way I 
>>>>>>>> am not
>>>>>>>> worried about destroying a client's domain while I am toying 
>>>>>>>> around.
>>>>>>>>
>>>>>>>> On 08/22/2014 10:34 AM, L.P.H. van Belle wrote:
>>>>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>>>>> Van: ryana at reachtechfp.com
>>>>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>>>>> Verzonden: vrijdag 22 augustus 2014 15:30
>>>>>>>>>> Aan: samba at lists.samba.org
>>>>>>>>>> Onderwerp: Re: [Samba] Proper sysvol replication solution...
>>>>>>>>>>
>>>>>>>>>> I know it is, on purpose. Copying the ACLs from one DC to
>>>>>>>>>> another is bad
>>>>>>>>>> because for some reason each DC may have a different ID for
>>>>>>>>>> the built-in
>>>>>>>>>> accounts. In fact I believe it was Rowland who was saying 
>>>>>>>>>> something
>>>>>>>>>> about copying the "idmap.tdb" file over if you sync the ACLs.
>>>>>>>>>> My method
>>>>>>>>>> only copies over the files and directories so no ACLs get
>>>>>>>>>> screwy. Maybe
>>>>>>>>>> Rowland can reiterate what he said on the other thread.
>>>>>>>>>>
>>>>>>>>> Yes, your right, it can be different if you dont keep de the
>>>>>>>> default settings.
>>>>>>>>> Which should be ok. I dont have any reason to change the
>>>>>>>> acls on sysvol.
>>>>>>>>> And i only use my ADDC as DC and not as file server so i
>>>>>>>> dont have any idmap problems.
>>>>>>>>> And i only use windows users to access these shares.
>>>>>>>>> No linux users or what have access to these machines. only
>>>>>>>> me as linux user and windows user.
>>>>>>>>> and that are 2 different accounts. my linux user is only use
>>>>>>>> to maintain the system.
>>>>>>>>>> I actually did use rsync for a while and then realized I was
>>>>>>>>>> messing up
>>>>>>>>>> after several machines stopped being able to access the sysvol
>>>>>>>>>> share on
>>>>>>>>>> the second DC. When I ran "samba-tool ntacl sysvolcheck" it
>>>>>>>>>> would crash.
>>>>>>>>>> I ran a reset and all was good, until the next rsync. I
>>>>>>>>>> finally figured
>>>>>>>>>> it out and removed rsync and the rsync cron job.
>>>>>>>>>>
>>>>>>>>>> So tell me, why are you using rsync and unison? I am also 
>>>>>>>>>> trying to
>>>>>>>>>> figure out how your script figures out which way to use rsync
>>>>>>>>>> (DC1->DC2
>>>>>>>>>> or DC2->DC1) based on what changes are on each DC. In other
>>>>>>>>>> words, would
>>>>>>>>>> your script work if a directory and file was created on DC1 
>>>>>>>>>> AND a
>>>>>>>>>> directory was removed on DC2? Would the new stuff get copied
>>>>>>>>>> to DC2 and
>>>>>>>>>> would the directory be removed on DC1? Forgive me, but I am
>>>>>>>> not a bash
>>>>>>>>>> scripting god. I know the basics and can get my work done, but
>>>>>>>>>> I am not
>>>>>>>>>> understanding how your script does two-way stuff via the
>>>>>>>> rsync portion.
>>>>>>>>> Well normal questions.. so
>>>>>>>>> If a file is remove from sysvol on DC1 it removes it also from 
>>>>>>>>> DC2.
>>>>>>>>> If at the same time a folder or file was remove from DC2
>>>>>>>> then it does the same with it on DC1.
>>>>>>>>> just test it.. you see.
>>>>>>>>> And best tip of the day.. make a backup of your sysvol
>>>>>>>> before you start,
>>>>>>>>> when you rest things and you forget to remove the unison
>>>>>>>> hash files you can delete you sysvol..
>>>>>>>>> we need rsync to create the directory structure with
>>>>>>>> extended attributes,
>>>>>>>>> the (now) unison setup copies only the extened attributes on 
>>>>>>>>> files.
>>>>>>>>>
>>>>>>>>> cron does the sync every 5 min for me and you can also use
>>>>>>>> inotify for that.
>>>>>>>>> You can search for that one on the list.
>>>>>>>>>
>>>>>>>>> Hope answere this helps you.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 08/22/2014 02:21 AM, L.P.H. van Belle wrote:
>>>>>>>>>>> ah you didnt see my script...
>>>>>>>>>>>
>>>>>>>>>>> https://secure.bazuin.nl/scripts/3-setup-sysvol-bidirectional.sh 
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> You way is missing the ACL.. you need rsync with unison.
>>>>>>>>>>> its all in my script.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Greetz,
>>>>>>>>>>>
>>>>>>>>>>> Louis
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>>>>>>> Van: ryana at reachtechfp.com
>>>>>>>>>>>> [mailto:samba-bounces at lists.samba.org] Namens Ryan Ashley
>>>>>>>>>>>> Verzonden: vrijdag 22 augustus 2014 0:14
>>>>>>>>>>>> Aan: samba at lists.samba.org
>>>>>>>>>>>> Onderwerp: [Samba] Proper sysvol replication solution...
>>>>>>>>>>>>
>>>>>>>>>>>> I see the Samba guide suggests using rsync to keep sysvols in
>>>>>>>>>>>> sync, but
>>>>>>>>>>>> this poses a problem with ID's and it is only one-way. I 
>>>>>>>>>>>> have been
>>>>>>>>>>>> hesitant to suggest anything because of the flak I have
>>>>>>>>>> been getting,
>>>>>>>>>>>> but I do believe I have a much better solution that transfers
>>>>>>>>>>>> files via
>>>>>>>>>>>> SSH, is bi-directional (no more only editing group policy 
>>>>>>>>>>>> on one
>>>>>>>>>>>> server), and does NOT set UID/GID information. In other
>>>>>>>> words, it is
>>>>>>>>>>>> PERFECT for sysvol replication, and has been working on
>>>>>>>>>> several of my
>>>>>>>>>>>> domains for around a year and a half without a hitch. The
>>>>>>>>>>>> solution I am
>>>>>>>>>>>> proposing is to use unison, which also works on Windows and (I
>>>>>>>>>>>> think) Mac.
>>>>>>>>>>>>
>>>>>>>>>>>> The way I have unison working on my systems is to install
>>>>>>>>>>>> unison on all
>>>>>>>>>>>> DC's, which is required. You also need an SSH server and
>>>>>>>>>> client on all
>>>>>>>>>>>> DC's, but I assume most of you do anyway. Once they're
>>>>>>>>>>>> installed, it is
>>>>>>>>>>>> as simple as the command below. This will synchronizes changes
>>>>>>>>>>>> BOTH WAYS
>>>>>>>>>>>> without touching your UID/GID setup. If you're paranoid, 
>>>>>>>>>>>> you could
>>>>>>>>>>>> always do a sysvolreset when done though.
>>>>>>>>>>>>
>>>>>>>>>>>> unison -batch "/path/to/sysvol"
>>>>>>>>>> "ssh://dc02.domain.lan//path/to/sysvol"
>>>>>>>>>>>> If you do this at a command-line, it will prompt you for
>>>>>>>>>> your password
>>>>>>>>>>>> on the remote machine. This would prevent a cron job, but
>>>>>>>> I overcame
>>>>>>>>>>>> that as well. You can create an SSH key that does not require
>>>>>>>>>>>> a password
>>>>>>>>>>>> for the systems to use. This means you can now create a
>>>>>>>> cron job to
>>>>>>>>>>>> handle the replication every fifteen minutes or so. You
>>>>>>>>>> could also use
>>>>>>>>>>>> something like "incrond" to monitor for changes in the sysvol
>>>>>>>>>>>> and launch
>>>>>>>>>>>> unison as well, but I don't personally modify the sysvol 
>>>>>>>>>>>> often, so
>>>>>>>>>>>> replication every fifteen minutes works for me.
>>>>>>>>>>>>
>>>>>>>>>>>> To create an SSH key to allow password-less replication via
>>>>>>>>>> unison, do
>>>>>>>>>>>> the following.
>>>>>>>>>>>>
>>>>>>>>>>>> ssh-keygen -t dsa
>>>>>>>>>>>>
>>>>>>>>>>>> When it prompts for a file to save the key in, it should be
>>>>>>>>>> your home
>>>>>>>>>>>> directory in a ".ssh" directory. I run as root, so this is
>>>>>>>>>>>> "/root/.ssh/id_dsa" for me. It will then prompt for a
>>>>>>>>>> password. Ignore
>>>>>>>>>>>> this and just press enter. It will ask you to verify the
>>>>>>>>>>>> password. Press
>>>>>>>>>>>> enter again. If you enter a password here, it cannot run
>>>>>>>>>>>> without user input!
>>>>>>>>>>>>
>>>>>>>>>>>> Next, you need to copy the key to your other domain
>>>>>>>>>>>> controllers. You can
>>>>>>>>>>>> do so as follows. Note that my example is run as root.
>>>>>>>>>> Substitute your
>>>>>>>>>>>> user's path if needed.
>>>>>>>>>>>>
>>>>>>>>>>>> ssh-copy-id -i /root/.ssh/id_dsa.pub root at dc02.domain.lan
>>>>>>>>>>>>
>>>>>>>>>>>> Once that is done, login to the domain controller you copied
>>>>>>>>>>>> the key to
>>>>>>>>>>>> (in the example, dc02) and check "/root/.ssh/authorized_keys"
>>>>>>>>>>>> to verify
>>>>>>>>>>>> that the key was added and nothing unexpected is there. You
>>>>>>>>>>>> can do this
>>>>>>>>>>>> with "cat /root/.ssh/authorized_keys". You should see a key on
>>>>>>>>>>>> a single
>>>>>>>>>>>> line followed by the hostname of your primary domain
>>>>>>>>>> controller. If it
>>>>>>>>>>>> is there, they may now connect via SSH without a password!
>>>>>>>>>>>>
>>>>>>>>>>>> You may now copy the key to any other domain controllers in
>>>>>>>>>>>> your domain
>>>>>>>>>>>> so they trust the primary DC as well. After that, all that
>>>>>>>>>> is left is
>>>>>>>>>>>> the synchronization. I urge you to run the first 
>>>>>>>>>>>> synchronization
>>>>>>>>>>>> manually, like below.
>>>>>>>>>>>>
>>>>>>>>>>>> unison "/path/to/sysvol" 
>>>>>>>>>>>> "ssh://dc02.domain.lan//path/to/sysvol"
>>>>>>>>>>>>
>>>>>>>>>>>> Make sure everything looks good, synchronize it, then repeat
>>>>>>>>>>>> for each DC
>>>>>>>>>>>> on your domain. Once done, you can create cron jobs to sync
>>>>>>>>>>>> each server,
>>>>>>>>>>>> or use a script like mine below. This script is on my
>>>>>>>> primary DC. I
>>>>>>>>>>>> actually only have two DC's, but I added more as an example 
>>>>>>>>>>>> here.
>>>>>>>>>>>>
>>>>>>>>>>>> #!/bin/bash
>>>>>>>>>>>> SERVERLIST="dc02.domain.lan dc03.domain.lan dc04.domain.lan"
>>>>>>>>>>>> SVPATH="/path/to/sysvol"
>>>>>>>>>>>>
>>>>>>>>>>>> # Synchronize all of the domain controllers
>>>>>>>>>>>> for sLoop in ${SERVERLIST}
>>>>>>>>>>>> do
>>>>>>>>>>>>      unison -batch "${SVPATH}" "ssh://${sLoop}/${SVPATH}"
>>>>>>>>>>>> done
>>>>>>>>>>>>
>>>>>>>>>>>> exit 0
>>>>>>>>>>>>
>>>>>>>>>>>> Now set that script to run in a cron job and you're golden.
>>>>>>>>>> You could
>>>>>>>>>>>> also setup "incrond" on all of your DC's and have it call
>>>>>>>>>>>> unison to sync
>>>>>>>>>>>> the other DC's whenever a write happens in your sysvol,
>>>>>>>> but I do not
>>>>>>>>>>>> need such a thing and have not personally tried it,
>>>>>>>> though I have a
>>>>>>>>>>>> fellow IT lead who has and likes it. My crontab job entry is
>>>>>>>>>>>> listed below.
>>>>>>>>>>>>
>>>>>>>>>>>> 15 * * * * /root/sysvolsync.sh &> /dev/null
>>>>>>>>>>>>
>>>>>>>>>>>> I hope this helps somebody and if you see something wrong,
>>>>>>>>>>>> feel free to
>>>>>>>>>>>> let me know.
>>>>>>>>>>>> -- 
>>>>>>>>>>>> To unsubscribe from this list go to the following URL and 
>>>>>>>>>>>> read the
>>>>>>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>> -- 
>>>>>>>>>> To unsubscribe from this list go to the following URL and 
>>>>>>>>>> read the
>>>>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> -- 
>>>>>>>> To unsubscribe from this list go to the following URL and read the
>>>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>> Rowland do you remeber Michael Adam's patch here from april? Seems 
>>>>> it did not yet make it into an stable release.
>>>>
>>>> Yes, I vaguely remember it, but it did not go far enough (no 
>>>> disrespect meant Michael, I couldn't have got that far),  I 'think' 
>>>> it only dealt with 'BUILTIN' objects, but they are a small part of 
>>>> the well known RID's .
>>>>
>>>> Rowland
>>> Good point,
>>>
>>> An nss_idmap module or extention to nss_winbind which resolves 
>>> xidNumbers to objectSID's might do the trick. It is not neccesarry 
>>> to translate the SID into an name to solve the mapping problem.
>>>
>>> achim~
>>>
>> You mean like the reverse of what ever happens to create the entries 
>> in idmap.ldb ?
>>
>> Rowland
>>
> Yes,
>
> For your example in the other mail "getent group" would return
>
> S-1-5-32-544:*:3000000
>
> on the dc and
>
> S-1-5-32-544:*:2000
>
>
>
>
>
>
That would be good, but it would be better if was:

administrators:*:3000000

on the dc and

administrators:*:2000

Rowland