rsync mechanics question

[Tom Riley]

Matt,

Thanks for the reply. To clarify, I'm doing the following:

I have 2 mounts on a single computer the production mount is 100gigs
(/msgstore) and contains rough 17 million small files (email message
store), and a newly created 500g ufs file system (/mnt)

I'd like to minimize the downtime required to make the cut over of data,
so I'm doing low priority repeative rsync of the source data over to the
new 500g partition. Just prior to the downtime, I'll stop mail services
and do a final cold rsync and change the mount points.

Rsync is working fine, and doing its job.

However, the curiosity comes in with my source data taking up 86gigs of
data on a 100g partition, and as the copy progresses the destination
drive is reporting 240 gigs of usage.

So as far as I can tell, rsync is working and the data integrity seems
good, it's simply taking up 2.5 times the space.

This crosses realms of expertise that I'm a bit light on, and am fast
coming up to speed on. I'm trying to determine if there is some mechanic
within the rsync process that could account for the used space. James
mentioned that rsync creates temp files which could account for double
disk usage, and I'm following up on that. 

A second possibility is that because the disk is so large that Solaris
is doing something funky with the minimum block size per inode
assignment that's causing small files to consume more space on a larger
sized file system.

Ideally, I'd prefer the data copied from /msgstore to consume roughly
the same amount of the new disk as the old so I can have 400gigs of
growth rather than 250gigs.

Make sense? Any thoughts or suggestions would be welcomed.

-Tom

> -----Original Message-----
> From: hashproduct at gmail.com [mailto:hashproduct at gmail.com] On Behalf
Of
> Matt McCutchen
> Sent: Wednesday, May 09, 2007 2:56 PM
> To: Tom Riley
> Cc: rsync at lists.samba.org
> Subject: Re: rsync mechanics question
> 
> On 5/9/07, Tom Riley <axtjr at uaa.alaska.edu> wrote:
> > I've been using rsync (2.6.9) to migrate a 90g message store volume
and
> I'm
> > running into some interesting results.
> 
> Please be more specific about what is going wrong.  If you get an
> error message, please send the exact text.  If rsync is successful,
> what does it do that you didn't expect/want?
> 
> > Does rsync copy files at a file copy level or is it attempting to do
> some
> > block level copying?
> 
> Rsync works at file level, not block level.  (Of course, if a source
> file is itself a filesystem image, then one could say that rsync works
> at block level for that image.)
> 
> Matt