Can i prioritize my jobs with RSYNC ?

[Matt McCutchen]

On Thu, 2005-10-20 at 16:28 +0000, Amit Sharma wrote:
> My first query:
> 1. Can we make some sort of queuing procedure alongwith or through RSYNC. 
> Like in which we can provide priorities to particular JOBs in a queue. Even 
> if we are not able to make a seperate system for queuing, Is there a way to 
> change the priorities for my Jobs ( Like by default,  jobs are processed  
> alphabeticaly or numerically & that is the default priority order).

When you start an rsync, you might be able to get it to transfer files
in an order other than the default by using "--files-from" and supplying
a list of the files in the desired order.

> But what if i have been left with another 600 files to transmit & i want 
> last 100 to come at top of this queue or i want these last 100 jobs to be 
> transmitted on urgent priority. How to assign priorities to jobs.

Unfortunately, rsync is designed to make a list of all the files that it
will transfer at the very beginning and then send them one by one in
order.  There's no way to add to or reorder the list of files in a
running rsync.  Changing this would probably require a lot of work.

What you /can/ do is suspend the first rsync and start another to
transfer the 100 new files.  You might also find it useful to limit the
bandwidth used by the first rsync with "--bwlimit".

> Second query:
> 2. Is there a way to confirm online(not through log files) what exactly has 
> been transmitted & what is left to be transmitted. (For example we have the 
> --progress & -vvv verbosity options for telling me what exactly is happening 
> at this moment)

The options you mentioned are your best bet for monitoring what files
rsync has transmitted.  Consider using "--itemize" too.  I don't know of
any way to have rsync tell you what files are left, but when verbosity
is at level 3, rsync outputs its entire file list at the beginning of
the transfer in "make_file(...)" lines.  You could write a program to
read the file list, cross files off as rsync reports that they were
successfully transferred, and report remaining files on demand.

> Third query:
> 3. Also i wanted to know , is it possible for RSYNC to transmit more than 1 
> files at a same time ?

No.  If you want to transmit several files at once, use multiple rsyncs,
but be careful that several rsyncs don't work on the same file at once.
-- 
Matt McCutchen, ``hashproduct''
hashproduct at verizon.net -- http://mysite.verizon.net/hashproduct/