[Samba] Missing Samba documentation pages: migration
roma1390
roma1390 at gmail.com
Tue Dec 28 15:48:35 MST 2010
Hello,
There is a small document, which (I hope) useful to others samba user
to avoid mistakes, wrong path, that i take, to reach my goal.
This document is covered by license: xkcd501[12].
1. Target
2. Situation
2.1. Software versions
2.2. Software setup
3. Data migration
3.1. alternatives
3.1.1. robocopy
3.1.2. xcopy
3.1.3. net
3.1.4. smbfs/cifs
4. Migration
4.1. Start samba
4.2. ACL vs ACL
4.3. mount.cifs
4.4. copy using xcopy
4.5. copy using robocopy
4.6. documentation ...
4.7. net rpc share
4.8. NT_STATUS_MEDIA_WRITE_PROTECTED
4.9. share_info.tdb
4.10. ACL test using WINXP on DST
4.11. documentation ...
4.12. Searching for root
4.13. Samba works
4.14. Samba compiling
4.15. Limitations of net rpc share migrate
5. Migrations isn't migration
5.1. How to do better?
References
1. Target
=========
Goal is to migrate shares from old Pentium III with Windows
2003 (name it SRC) to Samba on Debian stable (name it DST).
2. Situation
============
Both servers are inside a domain. Primary (and the only one) DC
runs on Windows 2008 R2. Clients are mostly Windows Vista,
Windows 7, some Linux with Samba, some Windows XP machines.
SRC is used on during work hours, huge downtime isn't acceptable.
Stores about 2TB data on local SATA/IDE disks on NTFS file systems.
SRC has about 10 shares. And a lot of files (some times a lot of
small files)... Network card is 100Mbps, but copy test shows that
on large files SRC capable to make 50Mbps with near 100% CPU-kernel
time. On small files speed drops, drops near 10-20 times.
Having full time copy this can lead to 20 days [1] copy operation.
2.1. Software versions
----------------------
DST is run on Debian Stable 5.0 (lenny) with samba 2:3.2.5-4lenny13
with kernel 2.6.26-2-amd64 on Intel Core i7, data stored in raiserfs3
with ACL.
SRC with Windows 2003 running on Pentium 3. Data stored on several
NTFS partitions.
2.2. Software setup
-------------------
DST are joined into Domain. Domain name is MSHOME. This document doesn't
covers, domain join part, this is described inside The Official Samba
3.5.x HOWTO and Reference Guide[5].
There I will show that DST is correctly (I hope) configured, and how to
check that.
Is the winbindd deamon running?
DST# pidof winbindd
12111 12108 12072 12067 12064
Is winbind enabled in nsswitch.conf?
DST# cat /etc/nsswitch.conf | grep winbind
passwd: compat winbind
group: compat winbind
Is winbind working?
DST# getent passwd | grep administrator
administrator:*:10004:10013:Administrator:/home/MSHOME/administrator:/bin/false
DST# wbinfo -u | grep -i administrator
administrator
DST# wbinfo -U 10004
S-1-5-21-3103004317-3667232127-1157729718-500
DST# wbinfo -s S-1-5-21-3103004317-3667232127-1157729718-500
MSHOME+Administrator 1
What /etc/samba/smb.conf is?
DST# testparm -s 2>/dev/null| grep
'\(workgroup\|realm\|security\|password server\|idmap\|winbind\)'
workgroup = MSHOME
realm = MSHOME.LOCAL
security = ADS
password server = 10.0.0.250
idmap uid = 10000-20000
idmap gid = 10000-20000
winbind separator = +
winbind enum users = Yes
winbind enum groups = Yes
winbind use default domain = Yes
How share is made?
DST# cat /etc/samba/smb.conf | tail -n 4
[testshare]
path = /srv/testshare
browseable = yes
writable = yes
What permissions has test share?
DST# ls -ld /srv/testshare/
drwxrwxrwt+ 2 root root 112 2010-12-28 07:18 /srv/testshare/
DST# getfacl /srv/testshare/
# file: srv/testshare/
# owner: root
# group: root
user::rwx
group::rwx
other::rwx
Is this directory writable?
DST# id
uid=0(root) gid=0(root) groups=0(root)
DST# touch /srv/testshare/0.txt; echo $?
0
Is this directory writable by domain Administrator?
DST# su -s /bin/sh administrator -c 'id;touch
/srv/install/testshare/10004.txt;echo $?'
uid=10004(administrator) gid=10013(rtchsdomainservices)
groups=10002(BUILTIN+administrators),10006(domain admins),10008(schema
admins),10009(enterprise admins),10010(denied rodc password
replication group),10013(rtchsdomainservices),10014(group policy
creator owners),10018(rtc server local group)
0
3. Data migration
=================
To reach our goal, we need to move data from one server/storage to another
one, file by file, share by share. And by doing that we need to make sure
that:
- data is in sync
- ACL are copied
- minimal downtime.
Every service migration (how good it will be) needs data copy part. Lets
make a little discus about it.
3.1. alternatives
-----------------
After some unsuccessful try, and some searches I found some hints how to
do this "in samba way". [2] [3]
3.1.1. robocopy
^^^^^^^^^^^^^^^
Robocopy tool from Microsoft. Tool witch was made by some one inside of
corporation, and seems by mistake was discovered by sales people...
After that was some made the ugly part (added Robocopy GUI).
Distribution scheme of this tools is also confused. This can be found
inside resource kit (these days it's possible free download), or as
attachment from one of technet pages. Find download isn't easy.
To make this situation more worse, Robocopy is replaced with "better"
version: RichCopy.
This command must run from Windows. This can be a separate Windows
machine or SRC.
This tool is designed for recursive copy, so simple one file copy
is a big challenge. This tool can copy files, but if fails to
set correct ACL or owner doesn't issues any warnings.
3.1.2. xcopy
^^^^^^^^^^^^
native Windows from MS-DOS. Sadly this is deprecating in favor of
Robocopy [4].
This command must run from Windows. This can be a separate Windows
machine or SRC.
If fails to set ACL or owner stops copy with error.
3.1.3. net
^^^^^^^^^^
This is tool is from SAMBA suit, not from Windows. Has same name, but
different functionality. Bad name choice. This confuses Administrators,
users.
in [3] was suggested to use this command to copy data with ACL's.
DST# net rpc share migrate files testshare -S SRC --acls --attrs \
--timestamps -v -U Administrator%password
This command must run from Linux. This can be a separate Linux
machine or DST.
if something is wrong, no message is produced. If You aren't samba
developer, then options -d 10 can't help You too...
3.1.4. smbfs/cifs
^^^^^^^^^^^^^^^^^
Sadly but this seems to be the right and simples way doesn't work.
Author of this document tried many times and many ways but without
any success get acl's listed after mount share from SRC on DST.
smbclient can fetch ACL's from Windows server, but this isn't enought,
becaues smbclient can't be used with rsync, pax.
4. Migration
============
4.1. Start samba
----------------
This part was easy, aptitude install samba winbind, some work, to join
into domain test config, add some shares.
4.2. ACL vs ACL
---------------
ACL isn't portable as it's can be. Seems that posix guys and Windows
people has a different and incompatible view to world[6].
4.3. mount.cifs
---------------
Now is need to copy files from Windows to Samba. First idea was to
mount windows share localy and use rsync or pax copy whole files with
ACLS, sadly linux cifs client doesn't export this info.
4.4. copy using xcopy
---------------------
this was found on list[2]. This fails in some strange ways. There was any
helpful message (googlable token), so tried next method.
Login into both servers:
WINXP>net use \\SRC\testshare /user:Administrator *
Type the password for \\SRC\testshare:
The command completed successfully.
WINXP>net use \\DST\testshare /user:Administrator *
Type the password for \\DST\testshare:
The command completed successfully.
copy using xcopy
WINXP>xcopy \\SRC\testshare\test.txt \\DST\testshare\test.txt /E /H /R /K /O
Does \\DST\testshare\test.txt specify a file name
or directory name on the target
(F = file, D = directory)? F
\\SRC\testshare\test.txt
File creation error - This security ID may not be assigned as the owner of this
object.
4.5. copy using robocopy
------------------------
C:\cygwin\home\rokas>robocopy \\SRC\testshare\folder1
\\SRC\testshare\foo /E /COPY:DATSO /DCOPY:T
-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows :: Version XP026
-------------------------------------------------------------------------------
Started : Tue Dec 28 22:21:19 2010
Source : \\SRC\testshare\folder1\
Dest : \\SRC\testshare\foo\
Files : *.*
Options : *.* /S /E /COPY:DATSO /DCOPY:T /R:1000000 /W:30
------------------------------------------------------------------------------
New Dir 1 \\SRC\testshare\folder1\
2010/12/28 22:21:19 ERROR 5 (0x00000005) Copying NTFS Security to
Destination Directory \\SRC\testshare\folder1\
Access is denied.
New File 2.7 m file.exe
2010/12/28 22:21:19 ERROR 5 (0x00000005) Copying NTFS Security to
Destination Directory \\SRC\testshare\folder1\
Access is denied.
------------------------------------------------------------------------------
Total Copied Skipped Mismatch FAILED Extras
Dirs : 1 1 0 0 0 0
Files : 1 0 0 0 1 0
Bytes : 2.79 m 0 0 0 2.79 m 0
Times : 0:00:00 0:00:00 0:00:00 0:00:00
Ended : Tue Dec 28 22:21:19 2010
4.6. documentation ...
----------------------
In [7] is recommended first migrate users. But this isn't our target. And this
workaround seems to be a very big workaround. This method is also
recommended[3].
4.7. net rpc share
------------------
DST# net rpc share migrate shares testshare -S SRC -U administrator%password
DST#
as from unix world[*], this means: command completed successful.
but:
DST# find /srvtestshare/
/srv/testshare/
shows that seems nothing changed.
same command with -d 10 gives not helpful (not googable token) message:
[2010/12/28 15:42:19, 10] rpc_client/cli_pipe.c:rpc_api_pipe(893)
rpc_api_pipe: Remote machine SRC pipe \srvsvc fnum 0x4028 returned 24 bytes.
srvsvc_NetShareGetInfo: struct srvsvc_NetShareGetInfo
out: struct srvsvc_NetShareGetInfo
info : *
info : union srvsvc_NetShareInfo(case 502)
info502 : NULL
result : WERR_NET_NAME_NOT_FOUND
[2010/12/28 15:42:19, 1] utils/net_rpc.c:run_rpc_command(181)
rpc command function failed! (NT_STATUS_UNSUCCESSFUL)
other command was also tried:
DST# net rpc share migrate files testshare -S SRC
DST# net rpc share migrate files testshare -S SRC --acls
DST# net rpc share migrate files testshare -S SRC --acls --attrs
DST# net rpc share migrate files testshare -S SRC --acls --attrs --timestamps
DST# net rpc share migrate files testshare -S SRC --acls --attrs
--timestamps --destination DST
change "add share command" in smb.conf
other combinations ....
* - be silent if ok, only report errors.
4.8. NT_STATUS_MEDIA_WRITE_PROTECTED
------------------------------------
after some above unsuccessful tries NT_STATUS_MEDIA_WRITE_PROTECTED
appears.
Dmesg was clean, DST fs wasn't RO, local permissions was OK, smb.conf
also, other share with same config works ok, testshare fails to WINXP
create new file. So what is difference between two identical directories
on Linux filesystem, and same smb.conf files.
Tried with no success:
- reconnect WINXP client shares
- remove and recreate share directories
- recheck smb.conf share
- restart smb processes
- all above in other random orders.
All tests shows that this is Samba fault. But why?
DST# smbcontrol debug all:11
didn't help.
4.9. share_info.tdb
-------------------
don't ask how this came to my head, maybe this was effect of long
samba HowTo readings...
aptitude install tdb-tools
DST# tdbdump /var/lib/samba/share_info.tdb
{
key(13) = "INFO/version\00"
data(4) = "\02\00\00\00"
}
{
key(18) = "SECDESC/testshare\00"
data(XX) = "..... some binary data ......"
}
DST# tdbtool --help
Could not open --help: No such file or directory
tdb> ^D
DST# man tdbtool
No manual entry for tdbtool
See 'man 7 undocumented' for help when manual pages are not available.
of course who uses documentation? Only sissies... Real admins don't read
documentation, they copy paste "rm -rf /" from internet without check
on man page what this does...
so remove this bogus entry
DST# tdbtool /var/lib/samba/share_info.tdb delete "SECDESC/testshare\00"
4.10. ACL test using WINXP on DST
--------------------------------
seems we back to beginning. No more strange fails.
More information about the samba
mailing list