[Samba] Difference between real and reported disk usage

[Berni Ced]

Hi Bjoern and thank you for your reply.

Bjoern Baumbach <bb <at> sernet.de> writes:
> This is by design.
> With the strict allocate = yes Samba stops creating sparse files. If you
> copy a sparse file with a real size of e.g. 100MB and a
> reserved/allocated size of 1GB to a server with "strict allocate = yes",
> the server writes 900MB trailing zeros to the file 

Ok, ok, but from a user perspective i'm still not sure that there isn't
a bug somewhere. I try to concisely explain my points:

1) smb.conf man page doesn't mention that space allocation could bring
to a file larger than its effective size. Rather, the man page say:
---
When this is set to yes the server will change from UNIX behaviour of
not committing real disk storage blocks when a file is extended to the
Windows behaviour of actually forcing the disk system to allocate real
storage blocks when a file is created or extended to be a given size.
---
The key concept is "allocate *real* storage blocks".
I thought that Samba, by default, can optimize the space allocated,
using the sparse file feature to make some file occupy less space than
what is really needed. It was really a surprise when i get "no space
left on device" copying 45 GB of data on a 210 GB partition!

2) From smb.conf man page i've interpreted that if you use an extent
based filesystem like ext4 or xfs, "strict allocate = yes" is something
good, recommended, with many merit and apparently no big defects.

3) The behaviour is not (as can be guessed) the same with every clients.
This is because every client can have different space allocation policy
but also because, mmh, there can be a bug or something not handled
somewhere?
For example, i've analized the data copied with strict allocation
activated and i've found that Windows Server 2008 make files space
allocation in 1024 kiB blocks. This means that if you copy 1025 kiB,
that file will effectively occupy 2048 kiB. To be more precise the real
file occupation is a little (4 byte) more, but i think it's due to
linux filesystem overhead like extended attributes. But if i repeat the
same file copy from Windows 2000 Pro, i get no space overhead and the
file space results are the same obtained without strict allocate.
It could be interesting to know with Windows 7 or 8...
Maybe it's something that samba could manage to avoid the reported
overhead?

4) Even if it resolves some problems (i've heard with quota), how can
someone plan the samba share size or manage its free space if the
space allocation is such not predictable. The administrator have to
know every possible client that can and will connect to its samba
share. In other words, the more the environments is heterogeneous, the
worst.


If someone wants to do some test, can follow these steps:
1) Create a 1 MB file:
   dd if=/dev/urandom of=001M.txt bs=1024k count=1
2) Create a 1 MB + 1 kB:
   dd if=/dev/urandom of=001M1k.txt bs=1025k count=1
3) Repeat steps 1-2 creating 2MB, 3MB, 10MB and 2M+1k, 3M+1k, 10M+1k.
4) Copy the files on a Windows system of your choice.
5) Prepare a samba share configured with "strict allocate = yes":
6) Copy the file created before to the samba share using Windows.
7) Check the real file occupation with "ls -lks" or "du -s".

Cesare.