How to make big MySQL database more diffable/rsyncable? (aka
rsyncing big files)
martin at meltin.net
Tue Jul 7 21:15:18 GMT 2009
>>>>> "Krzysztof" == Krzysztof Nosek <krzysztof.nosek at techland.pl> writes:
Krzysztof> First, I was dumping the database with mysqldump to an
Krzysztof> uncompressed dump and rsyncing it. Such a dump would
Krzysztof> use some 34G of disk space. [...]
Some simple suggestions in addition to the one about experimenting
with block size:
* Does the output of mysqldump compress well with gzip?
If so, you could try compressing it with a version of gzip that
supports the --rsyncable option. The version in Debian or Ubuntu
Linux supports this option.
* If you're confident that you have a good line-oriented dump, how
does diff cope when comparing 2 dumps?
rsync and diff obviously use very different schemes for figuring out
what is different. However, sometimes doing a diff will show you
something obvious that you have missed. Obviously you'll need 68GB
of free disk to be able to try that!
* If you're not telling rsync to transfer just a single file, do the
filenames match at both ends of the transfer?
You not doing something like creating a dump with the date in the
filename, are you?
Sorry, no rocket science there... ;-)
peace & happiness,
More information about the rsync