<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Ok, so i did some tests.<div class=""><div class="" style="font-family: TimesNewRomanPSMT;"><div class=""><div class=""><span class="" style="font-family: AndaleMono; font-size: 13px; float: none; display: inline !important;">find /path </span><span class="" style="font-family: AndaleMono; font-size: 13px; float: none; display: inline !important;">-type f -ls > /dev/null</span></div></div></div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">First on my local SSD disk (1.9 millions files) :</div><div class="">1 find : </div><div class="">real<span class="Apple-tab-span" style="white-space:pre"> </span>2m16.743s<br class="">user<span class="Apple-tab-span" style="white-space:pre"> </span>0m7.607s<br class="">sys<span class="Apple-tab-span" style="white-space:pre"> </span>0m45.952s<br class=""><br class=""></div><div class="">10 concurrent finds (approx same results for each) :</div><div class="">real<span class="Apple-tab-span" style="white-space:pre"> </span>4m48.629s<br class="">user<span class="Apple-tab-span" style="white-space:pre"> </span>0m11.013s<br class="">sys<span class="Apple-tab-span" style="white-space:pre"> </span>2m0.288s<br class=""><br class=""></div><div class="">Almost double time is somehow logic.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Now same test on my server on the iSCSI disk (when there is no other activity) (2.8 millions files) :</div><div class="">1 find :</div><div class=""><div class="">real<span class="Apple-tab-span" style="white-space:pre"> </span>38m54.964s</div><div class="">user<span class="Apple-tab-span" style="white-space:pre"> </span>0m35.626s</div><div class="">sys<span class="Apple-tab-span" style="white-space:pre"> </span>4m33.593s</div></div><div class=""><br class=""></div><div class="">10 concurrent finds :</div><div class=""><div class="">real<span class="Apple-tab-span" style="white-space:pre"> </span>76m34.781s</div><div class="">user<span class="Apple-tab-span" style="white-space:pre"> </span>0m47.848s</div><div class="">sys<span class="Apple-tab-span" style="white-space:pre"> </span>5m42.034s</div></div><div class=""><br class=""></div><div class="">The difference is not crazy. But the find itself takes so much time !!!!!</div><div class="">I now see i have a real issue on that server. Transfer time is not a problem, but access time seems to be terribly slow.</div><div class=""><div><br class=""><blockquote type="cite" class=""><div class="">Le 21 mars 2018 à 16:59, Jayce Piel <<a href="mailto:jayce.piel@gmail.com" class="">jayce.piel@gmail.com</a>> a écrit :</div><br class="Apple-interchange-newline"><div class=""><span style="caret-color: rgb(0, 0, 0); font-family: TimesNewRomanPSMT; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Thanks for the answer.</span><div class="" style="caret-color: rgb(0, 0, 0); font-family: TimesNewRomanPSMT; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">I will do some tests of the stat() thing at a time when there is nothing else running.<br class=""><div class=""><br class=""></div><div class="">For the compression i tried to find the lowest common factor between the clients and the server. Server is older for now.</div><div class="">I used to use -c arcfour-128 before it was no more an option.</div><div class=""><br class=""></div><div class="">The 2 ciphers you are mentionning are available on the Clients but not on the server, sadly.</div><div class="">But i keep this in mind for when i will upgrade the server (or move the destination backups).</div><div class=""><br class=""></div><div class=""><br class=""><blockquote type="cite" class=""><div class="">Le 21 mars 2018 à 16:39, Kevin Korb via rsync <<a href="mailto:rsync@lists.samba.org" class="">rsync@lists.samba.org</a>> a écrit :</div><br class="Apple-interchange-newline"><div class=""><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">When rsync has a lot of files to look through but not many to actually</span><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">transfer most of the work will be gathering information from the stat()</span><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">function call. You can simulate just the stat call with: find /path</span><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">-type f -ls > /dev/null</span><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">You can run one then a few of those to see if your storage has issues</span><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">with lots of stats all at once.</span><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Also, why -c aes128-ctr ? If your OpenSSH is current then the default</span><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">of<span class="Apple-converted-space"> </span></span><a href="mailto:chacha20-poly1305@openssh.com" class="" style="font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">chacha20-poly1305@openssh.com</a><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;"><span class="Apple-converted-space"> </span>is much faster. If your systems have</span><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">AES-NI in the CPU then<span class="Apple-converted-space"> </span></span><a href="mailto:aes128-gcm@openssh.com" class="" style="font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">aes128-gcm@openssh.com</a><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;"><span class="Apple-converted-space"> </span>is much faster. If your</span><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">OpenSSH is too old for chacha to be the default then aes128-ctr was the</span><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">default anyway.</span><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">On 03/21/2018 09:49 AM, Jayce Piel via rsync wrote:</span><br class="" style="caret-color: rgb(0, 0, 0); font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><blockquote type="cite" class="" style="font-family: AndaleMono; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br class="">Here are my options :<br class=""><br class="">/usr/local/bin/rsync3 --rsync-path=/usr/local/bin/rsync3 -aHXxvE --stats<br class="">--numeric-ids --delete-excluded --delete-before --human-readable<br class="">—rsh="ssh -T -c aes128-ctr -o Compression=no -x" -z<br class="">--skip-compress=gz/bz2/jpg/jpeg/ogg/mp3/mp4/mov/avi/vmdk/vmem --inplace<br class="">--chmod=u+w --timeout=60 —exclude=‘Caches' —exclude=‘SyncService'<br class="">—exclude=‘.FileSync' —exclude=‘IMAP*' —exclude=‘.Trash' —exclude='Saved<br class="">Application State' —exclude='Autosave Information'<br class="">--exclude-from=/Users/pabittan/.UserSync/exclude-list --max-size=1000M<br class="">/Users/pabittan/ xserve.local.fftir:./<br class=""><br class=""></blockquote></div></blockquote></div><br class=""><div class=""><div class="" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div class="" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div class="">-- </div><div class=""><div class="">Jayce Piel — <a href="mailto:jayce.piel@gmail.com" class="">jayce.piel@gmail.com</a><span class="Apple-converted-space"> </span> -- 0616762431</div><div class=""> Responsable Informatique F.F.Tir</div></div></div></div></div></div></div></blockquote></div><br class=""><div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">-- </div><div class=""><div class="">Jayce Piel — <a href="mailto:jayce.piel@gmail.com" class="">jayce.piel@gmail.com</a> -- 0616762431</div><div class=""> Responsable Informatique F.F.Tir</div></div></div></div>
</div>
<br class=""></div></body></html>