The speed of copy is very very slow when much files in the folder

Nico Kadel-Garcia nkadel at gmail.com
Sat Apr 29 02:03:13 UTC 2017


On Fri, Apr 28, 2017 at 8:36 AM, Zhangxiaoxu via samba-technical
<samba-technical at lists.samba.org> wrote:
> Hi,
>
> 100,000 files in my share.
> When I copy some little files to the folder, the speed is very very slow.

In one directory, with no subdirectories? Maybe you should stop doing
this? No matter how clever the network file system, getting the
directory information sorted out and even sorting out permissions is
complicated by so many files in one directory. It can be workable,
especially if you're not trying to sort out a list of the files, but
it's not free.

> I add some logs in the smbd, the “get_real_filename_full_scan” function cast about 1s. It’s too slow.
> I think the function is used for comparing the dest name to the exist files to check whether the dest file is exist.
>
> When I add “case sensitive = yes” to the configuration file, the speed is much quickly.

Well, yes. Finding all the fiile names and sorting them is
computationally expensive, especially to verify whether a mixed case
file in that long list of files matches yours and there is a conflict.

> So, I think whether we has some method to improve the performance?
> Could you give us some suggestion about the case.

You just said one. "case sensitive - yes". The other is "don't put
that many files in one directory without splitting it out into
subdirectories".

> Environment:
>        Ubuntu 14.04
>        Samba 4.3.11
>
> Client:
>        Windows 7
>
> Configuration:
>        clustering = yes
>        ctdbd socket = /var/run/ctdb/ctdbd.socket
>    max protocol = SMB3
>    large readwrite = yes
>    idmap config *:range = 1000000-1999999
>    log level = 2
>    use sendfile = yes
>    store dos attributes = yes
>    acl_xattr:ignore system acls = yes
>    aio read size = 1024
>    deadtime = 10
>    aio write behind = true
>
> Could you add my bugzilla account to the samba domain.
> Email: zhang.xiaoxu at h3c.com<mailto:zhang.xiaoxu at h3c.com>
>
> Br,
> Zhang Xiaoxu.
>
> -------------------------------------------------------------------------------------------------------------------------------------
> 本邮件及其附件含有杭州华三通信技术有限公司的保密信息,仅限于发送给上面地址中列出
> 的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、
> 或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本
> 邮件!
> This e-mail and its attachments contain confidential information from H3C, which is
> intended only for the person or entity whose address is listed above. Any use of the
> information contained herein in any way (including, but not limited to, total or partial
> disclosure, reproduction, or dissemination) by persons other than the intended
> recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender
> by phone or email immediately and delete it!



More information about the samba-technical mailing list