Moving/merging a filesystem back into /

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

A little Gentoo specific info here...

On 12/02/13 16:24, Leen Besselink wrote:
> On Mon, Dec 02, 2013 at 02:31:23PM -0500, Charles Marcus wrote:
>> Hello,
>> 
>> I'm going to be moving a filesystem around, and was planning on 
>> using rsync to do it, so like to get some advice from those more 
>> experienced than I (both using rsync, and moving filesystems)...
>> 
>> I currently have a system that has a separate /usr on an LVM
>> partition.
>> 
>> I want to merge this back into the / (root) filesystem.
>> 
>> This is a production server (mail server, gentoo linux), so I'd 
>> really like to not brick this thing in the process.
>> 
>> What would the best arguments to use for making sure that
>> everything is preserved properly for a smooth transition?
>> 
> 
> I have some simple tips:
> 
>> Some suggestions on the gentoo list have been:
>> 
>> -a, or -axAHX, or -apogXx, or -PvasHAX
>> 
>> or should I go with a combined -apogsvxAHPX ?
>> 
> 
> I suggest you have a look at the output of rsync --help, it says:
> 
> -a, --archive               archive mode; equals -rlptgoD (no
> -H,-A,-X)
> 
> Clearly some of the ones you mention are already on that list.
> 
> There is no need to add them again.

Agreed completely.

> So what is stored in /usr ? Is there also a /usr/src ? or
> /usr/local that has any content ?

OP was talking about booting from a live media so no sub-mounts would
be mounted so this shouldn't matter.

> -A can be useful if you have ACLs, lots of systems don't.

ACLs are only supported if the filesystem is mounted with the acl
option so that is a simple way to check if it should be used (also you
will get errors if you try to write ACLs to a filesystem that isn't
mounted with that option).

> -X can be useful if you have extended attributes, lots of systems
> don't.

This is Gentoo specific but other distributions are moving this way.
Gentoo has the USE=caps and caps-ng settings which means to use file
capabilities flags instead of SetUID to root to perform certain
privileged tasks.  This means that programs like ping and traceroute
can be allowed to do the one privileged network function that they
need without chmoding them +s and forcing them to run with full root
privileges.  -X is needed to copy these flags.

> -H is for hardlinks, I don't usually see many of them around.

In Gentoo and others /usr/share/zoneinfo is full of hard links.  There
are others around the system too.  If you want to check your /usr for
hard links run this:
find /usr -mount -type f -links +1 -ls

> I've not seen a Gentoo system in a long time, but if /usr just
> contains installed packages then I think you won't see any of the
> above. But if you add them, they don't harm. It will just take more
> time and maybe use more memory.
> 
> For example -H will use more memory if I'm not mistaken. If you
> don't have enough memory, it might crash.

- -H will use more memory according to how many hard links it finds.  If
it runs you out of memory then you have tons of them and not using -H
would probably run you out of disk space.

> But I don't think it is usually a problem these days. But if Gentoo
> builds all the packages in /usr/src andd lots of source still
> exist, it might be very large.
> 
> If you do have a /usr/src or /usr/local or something like that. Are
> they on the same filesystem ?
> 
> If not, you want -x to prevent them being copied as well.
> 
> You can obviously easily check if there is anything else mounted
> under /usr by checking the mount command and/or /etc/fstab
> 
> I've never used -s

- -s is --protect-args.  It is often used in scripts so they can handle
any weird path someone might tell the script to copy.  It is important
if you have spaces in your source or target parameter.

> 
> I don't see any reason for using -p or -P for partial files,
> because if copying doesn't succeed you'll probably just start over.
> There is no network to worry about.

- -P is also shorthand for --progress.  On a local only copy
- --whole-file is forced so --partial is irrelevant.

>> Or is all that necessary? Would -a be all I need?
>> 
>> Thanks for any comments/suggestions...
>> 
>> ********** OT *********
>> 
>> If anyone cares to comment on the steps I've settled on, I'd 
>> appreciate that too:
>> 
>> 1. Boot off of the latest gentoo LiveDVD
>> 
>> 2. Mount /
>> 
>> mount /dev/sda3 /mnt/gentoo/
>> 
>> There should already be a /usr directory where it was being
>> mounted before, right? If not, then I guess I create it with
>> root:root 755 permissions.
>> 
>> 3. Mount old /usr to be moved/merged
>> 
>> vgscan vgchange -a y mount /dev/vg/usr /mnt/gentoo/oldusr
>> 
>> 4. Copy /oldusr to /usr
>> 
>> rsync -a? /mnt/gentoo/oldusr/ /mnt/gentoo/usr/
>> 
> 
> When using rsync from an other machine, which uses other
> uids/gids, I would always recommend to add:
> 
> --numeric-ids

If networking were involved I would completely agree with this.  If
the live CD has duplicate usernames with different numbers badness can
happen.

>> Are the trailing slashes required/important/necessary?
>> 
> 
> Yes, you need to use them when you are working with directories.
> 
>> Which arguments should I use?
>> 
>> 5. Edit /etc/fstab and comment/remove the /usr line
>> 
>> nano -wc /mnt/gentoo/etc/fstab
>> 
>> #/dev/vg/usr       /usr        reiserfs        noatime         0
>> 0
>> 
>> 6. Unmount mounted filesystems
>> 
>> umount /mnt/gentoo/oldusr umount /mnt/gentoo
>> 
>> 7. Reboot into new system
>> 
> 
> Anyway, on most Linux distributions, moving the content of /usr
> shouldn't be a big event.
> 
> Because the system doesn't change the content of /usr it usually
> just contains programs, libraries and other files from installed
> packages.
> 
> Don't worry, to much.
> 
> If it's a mailserver, what you could do is:
> 
> - shutdown - unplug the network - do everything you need to do to
> copy the files and change fstab - check if everything starts up
> normally - plug network back in
> 
> If with the check something fails, reboot with the livecd again and
> fix any problems.
> 
> Because the mailserver isn't connected to the network, no mail was
> lots or bounced.
> 
> I assume this is a standalone mailserver that doesn't depend on an
> external database. A mailserver shouldn't bounce mail if DNS isn't
> working for a little while.
> 
> Something else you could consider is to not automatically start the
> server processes and first check that everything works. Don't
> forgot to enable them to start automatically again when everything
> is done ;-)
> 
>> Done?
>> 
>> --
>> 
>> Best regards,
>> 
>> */Charles/*
> 
> Hope that was useful for you.
> 
>> -- Please use reply-all for most replies to avoid omitting the
>> mailing list. To unsubscribe or change options:
>> https://lists.samba.org/mailman/listinfo/rsync Before posting,
>> read: http://www.catb.org/~esr/faqs/smart-questions.html
> 


One more thing I didn't mention before...

Since /usr is pretty static you could "prime the copy" by doing the
rsync initially to /usr.new while the system is running.  Then you
would really only have an mv and and fstab edit to do from your live
environment.  You could probably even do those right before rebooting
if you are careful.

Even on something more dynamic like /home you can still do an initial
copy then update it from the live environment.

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlKdAH0ACgkQVKC1jlbQAQczuwCcCVm1u9jRaobQzY7E+DPe2gTj
08UAn3mEnICofT13odDji9pB7Ru5hkmQ
=xRLU
-----END PGP SIGNATURE-----