[clug] nfs issue

[George at Clug]

I too prefer " --delete-after " when working with rsync.


Recently I was having issues attempting to copy about 200 gb of files
ranging from 45 gb to very small using scp from one computer to
another over a single 1gbit network.


 There were symlinks in a number of folders and I wondered if these
were causing some confusion.


The issue that I was happening is that the copy process would be
working great, then for reasons I was not able to determine, the copy
would fail on one file such that the one file on the destination side
would keep growing until it filled the entire available disk space. On
the sending side, it appeared the copy process hung on the file but I
guess it was in some kind of loop, endlessly sending bytes, which were
being received by the destination and being written to disk. I would
stop the copy process, locate and delete the offending file that grew
too large, and restart the copy process.



The source was a physical computer, while the destination was a KVM
virtual computer. 



After several attempts of restarting the copy process where the scp
process would then fail on a different file, I decided the time and
effort being spent was no longer worth it, and I have moved on to
other tasks. I intended to try copying with rysnc and to copy from
physical to physical computers, one day, time permitting.


All the best at tracking down your issue.





On Monday, 20-05-2019 at 21:06 Paul Wayper via linux wrote:


On 5/20/19 7:20 PM, Eyal Lebedinsky via linux wrote:
> The source machine is both the nfs server and the local ntp server.
> 
> But even if time is slightly off, why would a file be missing at
times?

Maybe rsync thinks the file systems you're copying between are both
local?

I know that rsync does things differently if it's not talking to a
remote
rsync daemon (for either source or destination).  If I recall
correctly it
does entire file copies instead of incremental rewrites, on the
principle that
it's going to cost more to read the source and destination and write a
new
file than it is to simply copy the changed file over the destination.

This is pure speculation here, but I'm wondering if there's then some
weird
interaction between NFS's creation and locking semantics and rsync's
view of
the file system it's copying from.  I can't think what those might
be.

So it might be worth not doing an NFS mount but doing the rsync via
ssh.

I've also found --delete-after to be a bit better than --delete,
although I
think that was mainly with the --fuzzy option.

Hope this helps,

Paul

-- 
linux mailing list
linux at lists.samba.org
https://lists.samba.org/mailman/listinfo/linux