can't delete older file/directory links on another local partition after `snapshot' type backup script

Dalton Harvie daltonh at
Wed Mar 12 18:09:42 EST 2003

Thanks for looking at this, and developing rsync.  Really useful program that 
I rely on.  Bit of a newbie but I couldn't find anything on this in these 
archives.  Hopefully it's a really stupid question!

I'm trying to backup using cp -al and then rsync -auR --delete backup idea 
basically as per

I have a separate partition which I remount as rw, then cp -al yesterday's 
tree to a new one, say /todays/backup/directory.  Then

rsync -auR --delete --delete-excluded --ignore-errors --force 
--include-from=$includedfiles / /todays/backup/directory

to copy over any new files from the master (/) and delete any links to files 
that are no longer on the master.  This puts the root of master at 
/todays/backup/directory/ because of -R.

The problem: the links to old files and directories aren't getting deleted.  
As I understand --force shouldn't be needed, and I can't 
see why --ignore-errors would be necessary either in this case, but they 
don't make any difference.

It's all in a perl script (given below) run by cron.  Makes no difference 
whether run manually by root though.  I can remove directories or files from 
within the script using rm -r, so I don't think it is a permission or 
mounting problem?  I have it set up to use ssh by default (relevant?), but -e 
rsh doesn't make any difference.  I have tried doing this problem with small 
test directory trees on a single partition under a single user using the 
above command and that works fine.  Are there any file/directory deletion 
errors that could cause the deletion process to stop, even with 
--ignore-errors on?

rsync version 2.5.5, mandrake 8.2
Partition line from /etc/fstab: /dev/hdb1 /old ext3 defaults,ro 1 2

Any help appreciated --- I'm running out of disk space!

SCRIPT: relevant part is after 'now do rsync for real'

#!/usr/bin/perl -w
# ARCHIVE: script written to backup all info specified in archive.files
# Dalton Harvie, 24/2/03
# version 2: 26/2/03: include ls -l infor in archive.trans listing
# version 3: 10/3/03: including --force in rsync command to hopefully remove 
old directories
# wanted:
# 1) don't die without remounting /old as ro
# 2) gzip everthing and don't transfer unnecessary stuff if already there but 
# definitions


# define strings to define platform dependent functions

# $archivepart is a string specifying the partition which contains $archivedir

# $archivedir is the absolute path to archive structure
#$archivedir='/home/daltonh/tmp/old';  # defined globally
$archivedir='/old';  # so structure will be /old/old2003
# $log is a directory where the logfile archive.log should be stored

# $includedfiles is a filename which specifies in rsync format what files to 

# open logfile in $log directory and append new info

open(LOG, ">>$log/archive.log");

print LOG "archive script version $version called at ".(scalar 

# remount partition where the archive is in in rw mode
$systemcall="$MOUNT -w -o remount $archivepart";
(!(system($systemcall))) or do {print LOG "could not $systemcall\n"; die;};
# create backup directory if one hasn't already been specified

createdir($dir);  # call subroutine createdir to create the directory and 
possibly the tree too
print LOG "created directory for archive $dir\n";

# copy (hard link) files to directory from recent which has the previous tree 
(if it exists)


if (-e $recent and (-d $recent or -l $recent)) {  # have established that 
recent exists and that it's a link or directory

# copy (actually hard link) contents of $recent to new $dir, recursively, 
preserving atributes and copying but not following soft links
  $systemcall="$CP -al $recent/* $dir";  
  (!(system($systemcall))) or do {print LOG "could not $systemcall\n"; die;};

# now remove link ready for new one to be put in place

  unlink $recent or do {print LOG "could not unlink $recent\n"; die;};


# remove old archive.trans file if possible

if (-e "$dir/archive.trans") {unlink "$dir/archive.trans" or print LOG "could 
not unlink $dir/archive.trans\n";}
# rsync files to the new directory
# options: -a = -r (recursive) l (recreate symbolic links) ptgo (preserve 
permissions, time, group, ownership) D (copy devices)
#          -u = update
#          -R = relative filenames (eg /home/joeblogg -> old/home/joeblogg)
#          --delete = remove files from destination that aren't in source
#          --delete-excluded = remove files from destination that aren't 
included in transfer either
#          --include-from=$includedfiles = specify what to copy

# first check that $includedfiles exists

if (!(-f $includedfiles)) {print LOG "$includedfiles file does not exist\n"; 

# this bit modified for version 2
# open TEMP file and write out to this the transfer listing and some other 

open(TEMP, ">$dir.temp") or do {print LOG "could not open $dir.temp\n"; die;};

# assemble systemcall to run rsync for output purposes
$systemcall="$RSYNC -auRn --delete --delete-excluded 
--include-from=$includedfiles / $dir";  

print TEMP "archive script version $version called at ".(scalar 
print TEMP "rsync command: $systemcall\n";
print TEMP "archive directory $dir\n";
print TEMP "this file contains a listing of files that were transferred for 
this backup\n";
print TEMP ('_'x120)."\n";
print TEMP `$systemcall`;
print TEMP ('_'x120)."\n";

close TEMP;

# now, open TEMP again, this time for reading
open(TEMP, "<$dir.temp") or do {print LOG "could not open $dir.temp\n"; die;};
# open TRAN, the destination file for this info, for writing

open(TRAN, ">$dir.trans") or do {print LOG "could not open $dir.trans\n"; 

# loop through initial lines, reading from TEMP and writing to TRAN

while (<TEMP>) {
  print TRAN $_;
  if (/^building file list/) {last;}  # this indicates the next line will be 
file listings

# now loop through file names, using ls -l to give file specifics

while (<TEMP>) {
  if (/^wrote/) {print TRAN $_; last;}  # this indicates that we have 
finished the file listings
  print TRAN `$LS -l $_` or print TRAN "problem listing $_";

# write out final lines
while (<TEMP>) {
  print TRAN $_;

close TRAN;
close TEMP;

# get rid of TEMP

unlink "$dir.temp" or print LOG "could not unlink $dir.temp\n";

# now do rsync for real

# trying to work out why directories aren't deleted --- maybe permission 
problem? --- nah don't think so as rm -r works 
# shouldn't need --force or --ignore-errors according to me
$systemcall="$RSYNC -auR --delete --delete-excluded --ignore-errors --force 
--include-from=$includedfiles / $dir";  
(!(system($systemcall))) or do {print LOG "could not $systemcall\n"; die;};

# temp old
#$systemcall="$RSYNC -auR --delete --delete-excluded --force 
--include-from=$includedfiles / $dir";  
#(!(system($systemcall))) or do {print LOG "could not $systemcall\n"; die;};
# temp try just deleting something using this script --- works
#$systemcall="rm -r $dir/home/daltonh/balls/herbo/ir1";
#(!(system($systemcall))) or do {print LOG "could not $systemcall\n"; die;};


# if possible move transfer file to directory

if (!(-e "$dir/archive.trans")) {    # this is an unlikely possibility if 
archive.trans already exists
  $systemcall="$MV $dir.trans $dir/archive.trans";
  (!(system($systemcall))) or print LOG "could not $systemcall\n";

# create a new soft link from the new directory to $recent

if (!(-e $recent))  # true if $recent doesn't exist
  $systemcall="$LN -s $dir $recent";  
  (!(system($systemcall))) or do {print LOG "could not $systemcall\n"; die;};
  print LOG "$recent link cannot be made because of existing file\n";

# remount partition where the archive is in in ro mode
$systemcall="$MOUNT -r -o remount $archivepart";
(!(system($systemcall))) or do {print LOG "could not $systemcall\n"; die;};
print LOG "archive finished\n".('_'x80)."\n";

# subroutine to check and create backup directory tree
# new directory (complete path from root) is given by $dir

sub createdir

# set date variables
  $year=(localtime)[5]+1900;        # portable date functions


# loop through all required directories up to final checking and creating if 
# change directory permissions
  if (!(-d $archivedir)) {print LOG "archivedir $archivedir does not 
exist\n"; die;}

  foreach $dirc ($diryear,$dirmonth)
    if (!(-d $dirc)) {mkdir ($dirc, 0755) or do {print LOG "can't make $dirc: 
$! \n"; die;};}

# now loop through possible directory suffixes finding a directory that 
hasn't been created yet
  foreach $diradd ('','a'..'z','A'..'Z')

    $dirtry=$dirday.$diradd;              # add a suffix to the default day 

    if (!(-d $dirtry))                    # test if that directory exists
      mkdir ($dirtry, 0755) or do {print LOG "can't make $dirtry: $!\n"; 
      $dir=$dirtry;                       # must have made directory if we 
reach here

    if ($dir) {last;}             # if $dir has been set then a directory has 
been creted


# check that a directory actually was created
  if (!($dir))
    print LOG "ran out of directory suffixes when attempting to create 



More information about the rsync mailing list