pauses sync'ing between tmpfs and disk on Linux 2.4.x

[Steve Traugott]

Sounds like you're swapping -- what does free memory look like?

Steve

On Wed, Mar 23, 2005 at 03:24:51PM -0800, Ray Van Dolson wrote:
> I've set up a 1GB tmpfs filesystem on a system with a single IDE disk and
> 2GB's of memory.  I'm storing a large amount of RRD files (~300MB) on the
> tmpfs filesystem to make their generation a bit speedier... this part works
> great.
> 
> However, I want to rsync these files over from time to time to a directory on
> the local filesystem (same physical server).  I'm using rsync 2.6.4pre3 and
> am hoping to understand a bit better what is happening.
> 
> When I run rsync -av /path/to/tmpfs /path/to/diskdir things move along pretty
> fast for while, then there's a big pause... sometimes for up to 30+ seconds
> where nothing seems to be happening, but all IO on the system ceases (can't do
> anything in another xterm).  Then rsync starts moving along again for a while,
> and then pauses again... after about three such pauses it finishes the entire
> rsync process.
> 
> I'm wondering what I can do to speed things up... perhaps whatever processes
> that write to the tmpfs filesystem are fighting with rsync... but doesn't
> rsync just need read access?
> 
> I've also tried with -W (copy whole file) and used a smaller -B value.
> Anyways, I'll explain a bit about the curren scenario:
> 
> 1. Daemon on system receives data from remote devices.
> 2. Daemon calls rrdtool to write a .rrd file to the tmpfs filesystem.
> 3. rsync runs periodically (every 5 minutes) to sync up the tmpfs filesystem
>    with a directory on the local filesystem.
> 
> The tmpfs filesystem is a directory with about 100 directories inside of it,
> each containing rrd files.  I'm considering using a script to just rsync one
> of these subdirectories at a time over a period of time to "distribute" the
> load.
> 
> The main issue is that while rsync is running and enters these random 30
> second "pauses", no IO can happen and things get really backed up on the
> system and it ends up being faster for me to just dump the tmpfs completely
> and go back to writing directly to disk as slow as that was...
> 
> Any suggestions?
> 
> -- 
> Ray Van Dolson
> Linux/Unix Systems Administrator
> DigitalPath Networks
> -- 
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
> 

-- 
Stephen G. Traugott  (KG6HDQ)
UNIX/Linux Infrastructure Architect, TerraLuna LLC
stevegt at TerraLuna.Org 
http://www.stevegt.com -- http://Infrastructures.Org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.samba.org/archive/rsync/attachments/20050323/ceee7969/attachment.bin