<div dir="ltr">Yeah I know, there's more I need to do to optimize. That script is probably nearing ten years old. It's been running without a single problem so I never bothered to revisit it, even after updating machines, I just copy it over and keep in trucking.<br>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jan 22, 2013 at 4:14 PM, Kevin Korb <span dir="ltr"><<a href="mailto:kmk@sanitarium.net" target="_blank">kmk@sanitarium.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">-----BEGIN PGP SIGNED MESSAGE-----<br>
Hash: SHA1<br>
<br>
</div>Also, if you put dates and times in the file names instead of .01,<br>
.02, etc you don't have to do any mv's, you can easily tell when each<br>
backup was run, and ls can tell you which the newest and oldest are.<br>
<div class="im"><br>
On 01/22/13 18:12, Kevin Korb wrote:<br>
> That is the old way that pre-dates --link-dest. Instead of cp -al<br>
> daily.02 daily.01 you can do a mkdir daily.01 then an rsync ...<br>
</div><div><div class="h5">> --link-dest=../daily.02 daily.01<br>
><br>
> Rsync then doesn't need any --delete and you don't bother making<br>
> any hard links that will eventually be replaced. Plus the linking<br>
> happens while rsync is running so it is usually much faster.<br>
><br>
> On 01/22/13 18:10, Ashley M. Kirchner wrote:<br>
>> Joe, this is specific to having a backup with rsync. The way I<br>
>> use links for rsync is by not using the link (ln) command at<br>
>> all, but instead using cp's build-in -l (link) option. It looks<br>
>> something like this:<br>
><br>
>> 1) delete the oldest backup (simple 'rm' command) 2) shift the<br>
>> rest (with 'rm') by 1, so if I have a 7 day daily backup, it<br>
>> moves daily.06 -> 07, daily.05 -> 06, etc., etc. This<br>
>> effectively removed daily.01 as that is now .02. 3) make a linked<br>
>> copy of .02 back into .01, so cp -al daily.02 daily.01 - this<br>
>> links only, it doesn't physically copy anything. 4) run rsync<br>
>> against that .01 backup. I run rsync with --delete and<br>
>> --delete-excluded options in case something got removed from the<br>
>> live system, it gets deleted, but only from this newest backup.<br>
>> If I need to recover it 5 days from now, I still can by looking<br>
>> in daily.05.<br>
><br>
>> This setup works well in that only one folder will have actual<br>
>> physical files (and thus the largest of them all.) The rest of<br>
>> them are links back to it and are very small. The neat thing<br>
>> is, when a file gets deleted from the newest backup, .01, it<br>
>> breaks the link from the others, and the physical file remains in<br>
>> the next older backup, in this case .02, (till it "expires" out<br>
>> of the cycle.)<br>
><br>
>> I have 12 servers backing up gigabytes worth of data every night<br>
>> like this, maintaining anywhere between 3 to 6 weeks worth for<br>
>> each one. They're all going to a single 1 terabyte drive.<br>
><br>
>> A<br>
><br>
><br>
><br>
>> On Tue, Jan 22, 2013 at 3:31 PM, Franois <<a href="mailto:daitheflu@free.fr">daitheflu@free.fr</a><br>
>> <mailto:<a href="mailto:daitheflu@free.fr">daitheflu@free.fr</a>>> wrote:<br>
><br>
>> Hi Joe,<br>
><br>
>> If you want to understand hard-links, just take a look at<br>
>> Wikipedia : <a href="http://en.wikipedia.org/wiki/Hard_link#Example" target="_blank">http://en.wikipedia.org/wiki/Hard_link#Example</a><br>
><br>
>> I think it's pretty easy to understand.<br>
><br>
>> To understand how hard-links (and rsync) can help you make strong<br>
>> incremental backups, head over<br>
>> <a href="http://blog.interlinked.org/tutorials/rsync_time_machine.html" target="_blank">http://blog.interlinked.org/tutorials/rsync_time_machine.html</a><br>
><br>
>> Cheers,<br>
><br>
>> -- Franois<br>
><br>
><br>
>> Le mardi 22 janvier 2013 23:12:38, Joe a ￩crit :<br>
>>> Thanks for the reply. I know what hard and soft links are and<br>
>> have some<br>
>>> idea of how they relate to backup.<br>
>>><br>
>>> What I need is a tutorial on how all of that works with rsync.<br>
>>> I can see that there are a lot of considerations as to which<br>
>>> options to use for different situations and maybe some general<br>
>>> strategies on how to build something like an incremental or<br>
>>> differential backup.<br>
>>><br>
>>> I use rsync now on a one directory tree to one directory tree<br>
>> basis now,<br>
>>> but I'd like to have more than one backup on my backup device<br>
>>> without doubling the storage which is what hard links will<br>
>>> help me do once I know how they work in more detail.<br>
>>><br>
>>> Joe<br>
>>><br>
>>> On 01/22/2013 02:48 AM, Tony Abernethy wrote:<br>
>>>> This may help: (man ln)<br>
>>>><br>
>>>> A hard link to a file is indistinguishable from the original<br>
>>>> directory entry; any changes<br>
>> to a<br>
>>>> file are effectively independent of the name used to<br>
>>>> reference<br>
>> the file.<br>
>>>> Hard links may not normally refer to directories and may not<br>
>> span file<br>
>>>> systems.<br>
>>>><br>
>>>> Assuming you do many backups and many of the files do not<br>
>>>> change, hard links are your friend.<br>
>>>><br>
>>>> Backing up soft links: Do you back up the link or what the<br>
>>>> link points to? (Even that simple thing has interesting ways<br>
>>>> to get complicated.)<br>
>>>><br>
>>>> -----Original Message----- From:<br>
>>>> <a href="mailto:rsync-bounces@lists.samba.org">rsync-bounces@lists.samba.org</a><br>
>> <mailto:<a href="mailto:rsync-bounces@lists.samba.org">rsync-bounces@lists.samba.org</a>><br>
>>>> [mailto:<a href="mailto:rsync-bounces@lists.samba.org">rsync-bounces@lists.samba.org</a><br>
>> <mailto:<a href="mailto:rsync-bounces@lists.samba.org">rsync-bounces@lists.samba.org</a>>] On Behalf Of Joe Sent:<br>
>> Tuesday,<br>
>>>> January 22, 2013 1:32 AM To: <a href="mailto:rsync@lists.samba.org">rsync@lists.samba.org</a><br>
>>>> <mailto:<a href="mailto:rsync@lists.samba.org">rsync@lists.samba.org</a>> Subject: Is there a<br>
>>>> howto/tutorial on backups/rsync that covers<br>
>> the use<br>
>>>> of hard and soft links?<br>
>>>><br>
>>>> There have been a lot of posts on the list lately about<br>
>>>> issues<br>
>> with hard<br>
>>>> links. It has been very interesting, but I don't understand<br>
>>>> it very thoroughly. I haven't used hard links for anything<br>
>>>> yet. I've used symlinks - not for backups, of course - and<br>
>>>> have seen them get<br>
>> broken or<br>
>>>> deleted in backups.<br>
>>>><br>
>>>> Is there a tutorial anywhere that will explain how this<br>
>>>> works<br>
>> (assuming<br>
>>>> that the reader understands the basic concepts of backups<br>
>>>> and<br>
>> knows how<br>
>>>> to program, but doesn't really understand how to use links<br>
>>>> to create things like incremental or differential backups)?<br>
>>>> It seems like<br>
>> there<br>
>>>> are a lot of fine points to consider, some of which can<br>
>>>> really<br>
>> bite you<br>
>>>> if you don't take them into account.<br>
>>>><br>
>>>> I'm working on my own personal backup system using bash and<br>
>> rsync. When<br>
>>>> it's done it will be pretty good, but it would be *much*<br>
>>>> better if I rewrote it to have more backup versions using<br>
>>>> hard links to save<br>
>> space.<br>
>>>><br>
>>>> I am writing my own because (aside from learning a lot) I<br>
>>>> have<br>
>> only seen<br>
>>>> two types of backup utilities - those that are very<br>
>>>> simplistic<br>
>> and won't<br>
>>>> let me do what I want and those which are enterprise level<br>
>>>> and I<br>
>> can't<br>
>>>> figure out how to get them to do anything without extensive<br>
>> study. (I<br>
>>>> did experiment with areca (I think it uses rsync libraries<br>
>>>> under the hood) which would probably do everything I want,<br>
>>>> but I got stuck too many times and couldn't get enough<br>
>>>> support on their forum to keep going.)<br>
>>>><br>
>>>> TIA<br>
>>>><br>
>>>> Joe<br>
>>>><br>
>>>> -- Please use reply-all for most replies to avoid omitting<br>
>>>> the<br>
>> mailing list.<br>
>>>> To unsubscribe or change options:<br>
>>>> <a href="https://lists.samba.org/mailman/listinfo/rsync" target="_blank">https://lists.samba.org/mailman/listinfo/rsync</a> Before<br>
>>>> posting, read:<br>
>>>> <a href="http://www.catb.org/~esr/faqs/smart-questions.html" target="_blank">http://www.catb.org/~esr/faqs/smart-questions.html</a><br>
>> -- Please use reply-all for most replies to avoid omitting the<br>
>> mailing list. To unsubscribe or change options:<br>
>> <a href="https://lists.samba.org/mailman/listinfo/rsync" target="_blank">https://lists.samba.org/mailman/listinfo/rsync</a> Before posting,<br>
>> read: <a href="http://www.catb.org/~esr/faqs/smart-questions.html" target="_blank">http://www.catb.org/~esr/faqs/smart-questions.html</a><br>
><br>
><br>
><br>
><br>
><br>
><br>
<br>
- --<br>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~<br>
Kevin Korb Phone: <a href="tel:%28407%29%20252-6853" value="+14072526853">(407) 252-6853</a><br>
Systems Administrator Internet:<br>
FutureQuest, Inc. Kevin@FutureQuest.net (work)<br>
Orlando, Florida <a href="mailto:kmk@sanitarium.net">kmk@sanitarium.net</a> (personal)<br>
Web page: <a href="http://www.sanitarium.net/" target="_blank">http://www.sanitarium.net/</a><br>
PGP public key available on web site.<br>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~<br>
-----BEGIN PGP SIGNATURE-----<br>
Version: GnuPG v2.0.19 (GNU/Linux)<br>
Comment: Using GnuPG with Thunderbird - <a href="http://www.enigmail.net/" target="_blank">http://www.enigmail.net/</a><br>
<br>
</div></div>iEYEARECAAYFAlD/HVIACgkQVKC1jlbQAQd2OwCgwF8H2EjiCmKcTboG5rK8w7Vm<br>
Sv4AoIIknAaOcuMLOj9dKlykKV1Wvu1F<br>
=kLZP<br>
<div class="HOEnZb"><div class="h5">-----END PGP SIGNATURE-----<br>
--<br>
Please use reply-all for most replies to avoid omitting the mailing list.<br>
To unsubscribe or change options: <a href="https://lists.samba.org/mailman/listinfo/rsync" target="_blank">https://lists.samba.org/mailman/listinfo/rsync</a><br>
Before posting, read: <a href="http://www.catb.org/~esr/faqs/smart-questions.html" target="_blank">http://www.catb.org/~esr/faqs/smart-questions.html</a><br>
<br>
</div></div></blockquote></div><br></div>