Moving/merging a filesystem back into /

Charles Marcus CMarcus at Media-Brokers.com
Mon Dec 2 15:44:39 MST 2013


On 2013-12-02 4:49 PM, Kevin Korb <kmk at sanitarium.net> wrote:
> A little Gentoo specific info here...
>
> On 12/02/13 16:24, Leen Besselink wrote:
>> On Mon, Dec 02, 2013 at 02:31:23PM -0500, Charles Marcus wrote:
>>> Hello,
>>>
>>> I'm going to be moving a filesystem around, and was planning on
>>> using rsync to do it, so like to get some advice from those more
>>> experienced than I (both using rsync, and moving filesystems)...
>>>
>>> I currently have a system that has a separate /usr on an LVM
>>> partition.
>>>
>>> I want to merge this back into the / (root) filesystem.
>>>
>>> This is a production server (mail server, gentoo linux), so I'd
>>> really like to not brick this thing in the process.
>>>
>>> What would the best arguments to use for making sure that
>>> everything is preserved properly for a smooth transition?
>> I have some simple tips:
>>
>>> Some suggestions on the gentoo list have been:
>>>
>>> -a, or -axAHX, or -apogXx, or -PvasHAX
>>>
>>> or should I go with a combined -apogsvxAHPX ?
>>>
>> I suggest you have a look at the output of rsync --help, it says:
>>
>> -a, --archive               archive mode; equals -rlptgoD (no
>> -H,-A,-X)
>>
>> Clearly some of the ones you mention are already on that list.
>>
>> There is no need to add them again.
> Agreed completely.

Ok, so based solely on what is included in -a, the above could be 
potentially shortened to:

-apogsvxAHPX

Then, after reading the following, looks like it could ultimately be 
whittled down to:

-avxHX --numeric-ids

?

>> So what is stored in /usr ? Is there also a /usr/src ? or
>> /usr/local that has any content ?

Yes...

> OP was talking about booting from a live media so no sub-mounts would
> be mounted so this shouldn't matter.

There are no submounts.

>> -A can be useful if you have ACLs, lots of systems don't.
> ACLs are only supported if the filesystem is mounted with the acl
> option so that is a simple way to check if it should be used (also you
> will get errors if you try to write ACLs to a filesystem that isn't
> mounted with that option).

Apparently no acls.

/dev/sda1               /boot           ext2 noauto,noatime  1 2
/dev/sda2               none            swap sw              0 0
/dev/sda3               /               ext3 noatime         0 1
/dev/sda4               /backups        ext3 noatime         0 2
/dev/vg2/home           /home           reiserfs noatime         0 0
/dev/vg2/usr            /usr            reiserfs noatime         0 0
/dev/vg2/var            /var            reiserfs noatime         0 0

So, no -A...

>> -X can be useful if you have extended attributes, lots of systems
>> don't.

Dunno if I do or not... how would I tell?

Would including -X cause a problem if there are none?

> This is Gentoo specific but other distributions are moving this way.
> Gentoo has the USE=caps and caps-ng settings which means to use file
> capabilities flags instead of SetUID to root to perform certain
> privileged tasks.  This means that programs like ping and traceroute
> can be allowed to do the one privileged network function that they
> need without chmoding them +s and forcing them to run with full root
> privileges.  -X is needed to copy these flags.
>
>> -H is for hardlinks, I don't usually see many of them around.
> In Gentoo and others /usr/share/zoneinfo is full of hard links.  There
> are others around the system too.  If you want to check your /usr for
> hard links run this:
> find /usr -mount -type f -links +1 -ls

Ummm... that does find a lot of links, but they all seem to be soft 
links, not hard links?

Most are links to other files in the same directory.

Is -H only about hard links? Or all links?

Looks like a ton of soft links in /usr/bin too...

Sadly, this is an ancient system, and only has 2GB of RAM...

The issue with  memory is only about Hard links?

Any way to do a test to see if it will run out of memory during the 
actual run?

If it does, would cp run into the same problem?

>> I've not seen a Gentoo system in a long time, but if /usr just
>> contains installed packages then I think you won't see any of the
>> above. But if you add them, they don't harm. It will just take more
>> time and maybe use more memory.
>>
>> For example -H will use more memory if I'm not mistaken. If you
>> don't have enough memory, it might crash.
> - -H will use more memory according to how many hard links it finds.  If
> it runs you out of memory then you have tons of them and not using -H
> would probably run you out of disk space.
>
>> But I don't think it is usually a problem these days. But if Gentoo
>> builds all the packages in /usr/src andd lots of source still
>> exist, it might be very large.
>>
>> If you do have a /usr/src or /usr/local or something like that. Are
>> they on the same filesystem ?

Yes... and while /usr/src only contains the kernel sources, /usr/portage 
contains all of the other system/application sources. I could move that, 
but I'd rather not right now.

Total size for /usr isn't *huge*, but is about 13.5GB.

>> If not, you want -x to prevent them being copied as well.
>>
>> You can obviously easily check if there is anything else mounted
>> under /usr by checking the mount command and/or /etc/fstab
>>
>> I've never used -s
> - -s is --protect-args.  It is often used in scripts so they can handle
> any weird path someone might tell the script to copy.  It is important
> if you have spaces in your source or target parameter.

So, since I'm doing this manually, it isn't needed?

>> I don't see any reason for using -p or -P for partial files,
>> because if copying doesn't succeed you'll probably just start over.
>> There is no network to worry about.
> - -P is also shorthand for --progress.  On a local only copy
> - --whole-file is forced so --partial is irrelevant.

Ok, scratch -p and -P...

>> When using rsync from an other machine, which uses other
>> uids/gids, I would always recommend to add:
>>
>> --numeric-ids
> If networking were involved I would completely agree with this.  If
> the live CD has duplicate usernames with different numbers badness can
> happen.

So, to be safe, use --numeric-ids...?

>>> Are the trailing slashes required/important/necessary?
>> Yes, you need to use them when you are working with directories.
>>
>> Anyway, on most Linux distributions, moving the content of /usr
>> shouldn't be a big event.
>>
>> Because the system doesn't change the content of /usr it usually
>> just contains programs, libraries and other files from installed
>> packages.
>>
>> Don't worry, to much.
>>
>> If it's a mailserver, what you could do is:
>>
>> - shutdown - unplug the network - do everything you need to do to
>> copy the files and change fstab - check if everything starts up
>> normally - plug network back in
>>
>> If with the check something fails, reboot with the livecd again and
>> fix any problems.
>>
>> Because the mailserver isn't connected to the network, no mail was
>> lots or bounced.
>>
>> I assume this is a standalone mailserver that doesn't depend on an
>> external database. A mailserver shouldn't bounce mail if DNS isn't
>> working for a little while.
>>
>> Something else you could consider is to not automatically start the
>> server processes and first check that everything works. Don't
>> forgot to enable them to start automatically again when everything
>> is done ;-)
>>
>> Hope that was useful for you.

Yes, thanks to both of you!

-- 

Best regards,

*/Charles/*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20131202/75078496/attachment.html>


More information about the rsync mailing list