Rsync for program loading on embedded platforms

[Greger Cronquist]

Donovan Baarda wrote:

> The algorithm sacrifices CPU to minimise data transfer when updating it.
>
>For it to be worth it, you must already have similar data that needs
>updating, and data transfer must be expensive relative to CPU.
>
>Often with embedded systems you are loading programs from scratch, so
>there is nothing to "update". Even if you are updating "programs",
>compiled binaries are often very different for only minor source
>changes. It would be worth analysing your data to see if there are more
>application specific ways to minimise "updates" (like partitioning data
>into discrete parts that can be updated independently).
>
>  
>
Is that really true---that the binaries differ that much? Isn't that 
mostly due to relocating the different functions to other areas of the 
binary? Which, I guess, might be hard for in-place rsyncing. I just did 
a quick test with two different binaries using xdelta and rdiff and the 
uncompressed deltas were less than 10% of the original size (xdelta of 
course being the smallest). So I have hopes that some kind of 
diff-related (even if it means keeping the old binary on a PC which we 
do anyway for tracability reasons) might work. Depending, of course on 
the overwrite and cpu issues.

>The checksum size you can use is a function of the data size, block
>size, and your acceptable failure rate. There are threads where this has
>been analysed and the latest rsync incorporates dynamic checksum and
>block sizing based on that discussion. librsync doesn't have it yet, but
>it could easily be added.
>  
>
Please correct me if I'm wrong, but don't most of these threads deal 
with finding the optimal block size depending on the file size? For an 
embedded system we might well use small blocks just to be able to use, 
say, a 16 bit checksum that is much faster to compute. Or even 8 bit 
checksums for that matter. If I understand it correctly, rsync only ever 
uses one (well two if counting both the rolling and the md4 checksums) 
checksum implementation varying the block size it's calculated on.


>I'd be interested to hear of anyone using or contemplating the rsync
>algorithm on embedded systems.
>
>  
>
As of yet, this is more of an academic interest spurred by the annoying 
delays in the compile-download-test loop... :-)

/Greger