Rsync for program loading on embedded platforms
greger_cronquist at yahoo.se
Wed Jun 2 07:25:03 GMT 2004
Donovan Baarda wrote:
> The algorithm sacrifices CPU to minimise data transfer when updating it.
>For it to be worth it, you must already have similar data that needs
>updating, and data transfer must be expensive relative to CPU.
>Often with embedded systems you are loading programs from scratch, so
>there is nothing to "update". Even if you are updating "programs",
>compiled binaries are often very different for only minor source
>changes. It would be worth analysing your data to see if there are more
>application specific ways to minimise "updates" (like partitioning data
>into discrete parts that can be updated independently).
Is that really true---that the binaries differ that much? Isn't that
mostly due to relocating the different functions to other areas of the
binary? Which, I guess, might be hard for in-place rsyncing. I just did
a quick test with two different binaries using xdelta and rdiff and the
uncompressed deltas were less than 10% of the original size (xdelta of
course being the smallest). So I have hopes that some kind of
diff-related (even if it means keeping the old binary on a PC which we
do anyway for tracability reasons) might work. Depending, of course on
the overwrite and cpu issues.
>The checksum size you can use is a function of the data size, block
>size, and your acceptable failure rate. There are threads where this has
>been analysed and the latest rsync incorporates dynamic checksum and
>block sizing based on that discussion. librsync doesn't have it yet, but
>it could easily be added.
Please correct me if I'm wrong, but don't most of these threads deal
with finding the optimal block size depending on the file size? For an
embedded system we might well use small blocks just to be able to use,
say, a 16 bit checksum that is much faster to compute. Or even 8 bit
checksums for that matter. If I understand it correctly, rsync only ever
uses one (well two if counting both the rolling and the md4 checksums)
checksum implementation varying the block size it's calculated on.
>I'd be interested to hear of anyone using or contemplating the rsync
>algorithm on embedded systems.
As of yet, this is more of an academic interest spurred by the annoying
delays in the compile-download-test loop... :-)
More information about the rsync