[2.5.6] File table overflow

[jw schultz]

On Thu, Feb 27, 2003 at 10:21:00AM +0100, jarausch at igpm.rwth-aachen.de wrote:
> J.W. Schultz writes
> 
> > On Wed, Feb 26, 2003 at 10:04:33AM +0100, jarausch at igpm.rwth-aachen.de wrote:
> >> Hi,
> >> 
> >> on my linux box (2.4.21-pre4-ac4)
> >> I've used rsync to mirror a rather large /usr/local tree (4 Gb)
> >> After transmission of many files I get the error message
> >> rsync error: some files could not be transferred (code 23) at main.c(1045)
> >> 
> >> According to /usr/include/asm/errno.h the code 23 says
> >> File table overflow
> 
> > That isn't errno 23.  Code 23 is an internal error code
> > meaning "some files could not be transferred" In other words
> > the transfer was not complete.  Look for other error
> > messages before that one.
> 
> I have a log file of more than 154,000 lines. Searching
> for 'error' or 'transfer' doesn't show anything.
> So, how can I find out the reason for the failure.
> rsync has processed about 154,000 files, though.
> There is plenty of space in the target directory.

Unfortunately many of the error messages do not contain the
words error or warn.  Many of them do have a colons in them.

> It might even be that rsync has synced all files
> but because of the error is doesn't honour the
> --delete option, so it's hard to check if some files
> are missing.
> 
> If I delete a file on the (remote) source directory,
> the error occurs on file later.
> 
> So it looks like a limitation (buffer, list, etc) of
> rsync.

Looking a little closer, what that message means is that at
some time during the run a file I/O error happened.  It could
be a chmod or chown error, it could be a failure to delete a
file, even failure to read the secrets file can cause this
so you could have a perfectly good transfer.  I found over
200 different messages that set the flag.  Perhaps 20 or so would
cause an abort before any transfer.  I'd guess that at least
10 are mere warnings.

I suspect that this is another area that could use some
cleanup.  That message should really only show up if there
really are files that didn't transfer.

Here are 147 of them so you can get an idea of what i mean.

(see the rsync man page for an explanation)
Badly formed boolean in configuration file:
Badly formed line in configuration file: %s
Batch file %s open error: %s
Batch file %s read error: %s
Batch file %s write error: %s
ERROR: The remote path must start with a module name
ERROR: The remote path must start with a module name not a /
ERROR: buffer overflow in %s
ERROR: compress can not be used with
ERROR: destination must be a directory when copying more than 1 file
ERROR: file corruption in %s. File changed during transfer?
ERROR: hard links not supported on this platform
ERROR: module is read only
ERROR: out of memory in %s
ERROR: symbolic links not supported
ERROR: write-batch and read-batch
Empty section name in configuration file.
Error parsing options:
Failed to add a new service
Failed to close : %s
Failed to dup/close : %s
Failed to exec %s : %s
Failed.  Error returned from params.c:parse().
Global parameter %s found in service section!
IGNORING unknown parameter
Ignoring badly formed line in configuration file: %s
Integer overflow - attempted 64 bit offset
Invalid file index %d (count=%d)
Invalid file index %d in recv_files (count=%d)
Invalid gid %s
Invalid parameter name in config. file.
Invalid uid %s
Memory re-allocation failure.
No configuration filename specified.
RSYNC_PASSWORD environment variable ignored
Unexpected EOF in the configuration file: %s
Unexpected end-of-file at: %s
Unknown Parameter encountered:
Unknown socket option %s
WARNING: fnmatch FNM_PATHNAME is broken on your system
auth failed on module %s from %s (%s)
backup filename too long
bad response from proxy - %s
buffer overflow expanding %%%c - exiting
chdir %s failed
chown %s : %s
chroot %s failed
compression init failed
continuing without password file
continuing without secrets file
copy %s -> %s : %s
could not open password file
decompressor lost sync!
deflate on token returned %d (%d bytes left)
deflate returned %d
delete_file: opendir(%s): %s
delete_file: rmdir(%s) : %s
delete_file: unlink(%s) : %s
delete_one: rmdir %s: %s
delete_one: unlink %s: %s
error pipe failed in do_recv
failed to connect to %s: %s
failed to open %s, continuing : %s
failed to open lock file %s : %s
failed to read from proxy: %s
failed to set permissions on %s : %s
failed to set socket option %s: %s
failed to set times on %s : %s
failed to write to proxy: %s
falling back to RSYNC_PASSWORD environment variable.
filename too long
fork: %s
fstat %s : %s
fstat failed : %s
getaddrinfo %s: %s
getaddrinfo: %s %s: %s
getaddrinfo: bind address %s: %s
getpeername on fd%d failed: %s
inflate (token) returned %d
inflate flush returned %d (%d bytes)
inflate init failed
inflate returned %d (%d bytes)
io timeout after %d seconds - exiting
keep_backup failed %s -> %s : %s
keep_backup filename too long
link %s -> %s : %s
link_stat %s : %s
lseek failed in map_ptr
make_bak_dir chown %s : %s
make_bak_dir failed to set permissions on %s : %s
make_bak_dir stat %s : %s
malformed address %s
malformed mask in %s
max connections (%d) reached
memory allocation failure.
mkdir %s: %s
mknod %s : %s
mkstemp %s failed: %s
multiplexing overflow %d\n
name lookup failed for %s: %s
not a regular file (recv_files)
open %s: %s
open inbound socket on port %d failed:
opendir(%s): %s
out of memory at %s(%d)
overflow: l=%d
pipe failed in do_recv
pipe: %s
pop_dir %s : %s
protocol version mismatch - is your shell clean?
push_dir %s : %s
push_dir %s : %s (1)
push_dir %s : %s (4)
push_dir %s: %s
push_dir %s: %s (3)
read %s: %s
read error: %s
readlink %s: %s
receive_sums failed
recv_generator: mkdir
remote rsync does not support include syntax - aborting
rename %s -> %s : %s
rename %s to backup %s
robust_move failed: %s(%d)
rsync denied on module %s from %s (%s)
rsync error: %s (code %d) at %s(%d)
rsync: did not see server greeting
rsync: didn't get server startup line
rsync: server sent %s rather than greeting
rsync: unable to open configuration file
send_files failed on long-named directory %s
send_files failed to open %s: %s
server_recv: recv_file_list error
setgid %d failed
setgroups failed
setuid %d failed
stat %s : %s
syntax error - %s does not take a value
unexpected tag %d
unknown family %u
unlink %s: %s
using RSYNC_PASSWORD environment variable.
write %s: %s
write failed on %s : %s

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt