[Samba] Samba 3.0, fuse-hdfs and write problems

[alo alt]

Hi folks,

I research at the moment a connection over fuse and samba for
hadoop-cluster. Its my private playground, but I have some issues I can't
figure out why they happen.

I use RHEL5.7, packetlist:
rpm -qa|grep samba
samba-3.0.33-3.29.el5_7.4.x86_64
samba-common-3.0.33-3.29.el5_7.4.x86_64

2 servers, one provides samba-shares and one I use as client.

Samba-Server (HOST2):
FuseFS:
hadoop-fuse-dfs#dfs://HOSTNAME:9000 /123/hdfs fuse usetrash,rw 0 0

mount:
fuse on /123/hdfs type fuse
(rw,nosuid,nodev,allow_other,default_permissions)

the mount /123 is accessible over ssh, cp, rsnyc from any host. Now I was
using NFS for populating, but the performance was not as I expected. So I
switched to samba and get real good results for reading.

Cifs-Client(HOST1):
mount -t cifs -o username=BLUBB,domain=hdfs,rw,uid=104 //lHOST2/hdfs /test

config:

[global]
        workgroup = MYGROUP
        server string = Samba Server Version %v
        socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
        security = user
        smb passwd file = /etc/samba/smbpasswd
        #passdb backend = tdbsam


[hdfs]
        comment = hhh
        path = /123
        public = yes
        writable = yes
        printable = no
        create mask = 0744
        force user = hdfs
        force group = nobody

I want only hdfs exported ;)

read-performance with rsync:
rsync -av /test/hdfs/user/flume/weblogs/2011-12-07/ .

sent 20478888644 bytes  received 92606 bytes  17377158.46 bytes/sec
total size is 20475835998

(19GB, 16 MB/s)

find . -type f |wc -l
4665

plain over rsync -essh .... I get around 20mb/s, so I can live with. But
now the part I spent days without a solution:

I perform a lot of write tests. I have a 10GB plain file (dd if of), thats
I use for all write tests. Now, if I try to copy the file from host1 (cifs
client), which hangs without socket options at 128MB. The interest part
there ist - 129MB are the blocksize in my hadoop-cluster. I played with
socket options and now the copy stops at 728MB. I get a nmb-error in
messages:
Dec  8 14:38:42 HOST2 nmbd[12609]: [2011/12/08 14:38:42, 0]
nmbd/nmbd.c:terminate(58)
Dec  8 14:38:42 HOST2 nmbd[12609]:   Got SIGTERM: going down...
Dec  8 14:42:46 HOST2 kernel: INFO: task ls:12663 blocked for more than 120
seconds.

after the system freezes, HOST1 going into load 41 up. vmstat und iostat
say nothing, cp is going into state Z (zombie). Has anybody a advice? Could
be that the process is overloaded and die?

A problem with fuse I could imagine, because if I copy the same file over
scp from HOST1 => HOST2 into hdfs I get a speed of 41 MB/s without any
errors.

thanks,
 - Alex



-- 
Alexander Lorenz
http://mapredit.blogspot.com

*P **Think of the environment: please don't print this email unless you
really need to.*