[Samba] Administrator lost write privileges to sysvol (Can't add/edit anything using RSAT Tools)

[Darren Conte]

Hello all,

I have researched this for a few days and I am now crazy perplexed since I
have not found any listings that fully describes my particular issue.

Let me re-trace my steps in hopes it can lead to a solution or find a root
cause.

Task at hand: Employee ‘Rodolfo’ quits, so I need to create new user
‘Nicole’ in AD to replace him.

NOTE: Rodolfo was a delegated User who was placed in ‘Domain Admins’ 6
months ago to help install programs, as needed. But Nicole will not be
needing such privileges. (this is important in my steps below as I believe
by removing him from this Group the incorrect way is what caused the
havoc).

Samba 4.11.4 running on Ubuntu 18.04.3 (server.radicallaw.net)
Windows 10 Pro PCs for 5 employees: lawpc01w10.radicallaw.net,
lawpc02w10.radicallaw.net….etc.
No linux workstations.

Here is what I did:
    1. I RDP to a PC as SAMDOM\Administrator, I launch Active Directory for
Users and Computers (RSAT).
    2. I open user Rodolfo, and remove ‘Domain Admins’ group within his
User Account on the ‘Members of’ tab (I believe this is the BUG that ruined
sysvol for SAMDOM\Administrator permissions because SAMDOM\Administrator
belongs to this Group too). I clicked Apply, OK to close Rodolfo user
account screen.
    3. I right-click on Rodolfo user account and select COPY, and
successfully create Nicole username and password, and click OK to close.
Doing so, auto-edited the particular fields for her Profile and Home
Directory.
    4. I close Active Directory for Users and Computers and log off of PC
as SAMDOM\Administrator.
    5. I RDP back to the PC as SAMDOM\nicole to setup her desktop, icons,
and profile.
    6. I log off of RDP as SAMDOM\nicole.
       I then realized I misspelled her last name, not posted here for
privacy. So I need to return to Active Directory for Users and Computers to
edit her Last Name field.
    7. I RDP to the PC as SAMDOM\Administrator, I launch Active Directory
for Users and Computers (RSAT).
    8. I open Nicole but realize certain fields are gray and not editable.
I also realize that when I right-click on any User, that both COPY and NEW
are no longer listed, as they were in Step 3 above.
       I figure that sysvol permissions are the culprit.
    9. I open Computer Management, navigate to Shared Folders>sysvol and
attempt to re-apply the Windows ACLs permissions using Advance Security
Settings, but get the error
“sysvol\Policies\{7EE86BCF-DEED-4927-8B75-CF9A7051B451} Failure to
enumerate object in container. Access denied.
    10. After using various commands to try and remediate (shown below) I
now get the error ‘\\SERVER\sysvol Failed to enumerate objects in
container. Access denied. Even though SAMDOM\Administrator is the owner and
ACLs appear to be correct.

NOTE: As you can see, originally only the Policies\{7EE…} directory was
affected. But now, after attempting to remediate the permissions on the
entire sysvol directory, I feel that I made the problem worse. But, if Step
2 above is a known issue, it changed only this folder.

But either way, SAMDOM\Administrator does not have WRITE privileges anymore
to any of the RSAT tools needed to administer the domain. But thankfully,
all Win10 PCs are functioning as normal with Domain Users (logon scripts,
GPO all are getting applied as expected), so production of the office has
not been affected.


HERE ARE MY CONFIGURATIONS
NOTE: here is my actual [global] to compare to the testparm results below.
[global]
dns forwarder = 192.168.1.1
netbios name = SERVER
realm = RADICALLAW.NET
server role = active directory domain controller
workgroup = RADICALLAW
time server = Yes
reset on zero vc = yes

root at server:/usr/local/samba/etc# testparm
Load smb config files from /usr/local/samba/etc/smb.conf
Loaded services file OK.
Server role: ROLE_ACTIVE_DIRECTORY_DC

Press enter to see a dump of your service definitions

# Global parameters
[global]
dns forwarder = 192.168.1.1
passdb backend = samba_dsdb
realm = RADICALLAW.NET
reset on zero vc = Yes
server role = active directory domain controller
time server = Yes
workgroup = RADICALLAW
rpc_server:tcpip = no
rpc_daemon:spoolssd = embedded
rpc_server:spoolss = embedded
rpc_server:winreg = embedded
rpc_server:ntsvcs = embedded
rpc_server:eventlog = embedded
rpc_server:srvsvc = embedded
rpc_server:svcctl = embedded
rpc_server:default = external
winbindd:use external pipes = true
idmap config * : backend = tdb
map archive = No
vfs objects = dfs_samba4 acl_xattr

[sysvol]
path = /usr/local/samba/var/locks/sysvol
read only = No

[netlogon]
path = /usr/local/samba/var/locks/sysvol/radicallaw.net/scripts
read only = No

[Profiles]
csc policy = disable
directory mask = 0700
level2 oplocks = No
oplocks = No
path = /Profiles
read only = No

[Users]
csc policy = documents
force create mode = 0600
force directory mode = 0700
path = /Users
read only = No
veto files = .deleted/
vfs objects = recycle
recycle:touch = True
recycle:versions = True
recycle:keeptree = True
recycle:repository = .deleted/%U

[Shares]
create mask = 0777
csc policy = documents
directory mask = 0777
force create mode = 0777
force directory mode = 0777
path = /Shares
read only = No
veto files = .deleted/
vfs objects = recycle
recycle:touch = True
recycle:versions = True
recycle:keeptree = True
recycle:repository = .deleted/%U

[printers]
browseable = No
comment = All Printers
create mask = 0700
guest ok = Yes
path = /var/spool/samba
printable = Yes

Here are my ACLs on sysvol in both formats:
root at server:/usr/local/samba/etc# samba-tool ntacl get --as-sddl
/usr/local/samba/var/locks/sysvol
O:BAG:BAD:(A;OICI;0x001f01ff;;;BA)(A;OICI;0x001200a9;;;AU)(A;OICI;0x001f01ff;;;SY)(A;OICI;0x001200a9;;;SO)(A;OICI;0x001f01ff;;;BA)(A;OICI;0x001200a9;;;AU)(A;OICI;0x001f01ff;;;SY)(A;OICI;0x001200a9;;;SO)(A;OICI;0x001f01ff;;;LA)(A;;0x001f01ff;;;BA)(A;;0x001f01ff;;;BA)(A;;0x001200a9;;;WD)(A;OICIIO;0x001f01ff;;;CO)(A;OICIIO;0x001200a9;;;CG)(A;OICIIO;0x001f01ff;;;WD)

root at server:/usr/local/samba/etc# getfacl /usr/local/samba/var/locks/sysvol
getfacl: Removing leading '/' from absolute path names
# file: usr/local/samba/var/locks/sysvol
# owner: 3000000
# group: 3000000
user::rwx
user:root:rwx
user:3000000:rwx
user:3000001:r-x
user:3000002:rwx
user:3000003:r-x
group::rwx
group:3000000:rwx
group:3000001:r-x
group:3000002:rwx
group:3000003:r-x
mask::rwx
other::r-x
default:user::rwx
default:user:root:rwx
default:user:3000000:rwx
default:user:3000001:r-x
default:user:3000002:rwx
default:user:3000003:r-x
default:group::r-x
default:group:3000000:rwx
default:group:3000001:r-x
default:group:3000002:rwx
default:group:3000003:r-x
default:mask::rwx
default:other::rwx

wbinfo --uid-info to clarify its mappings to the correct Groups.
root at server:/usr/local/samba/etc# wbinfo --uid-info=3000000
BUILTIN\administrators:*:3000000:3000000::/home/BUILTIN/administrators:/bin/false
root at server:/usr/local/samba/etc# wbinfo --uid-info=3000001
BUILTIN\server operators:*:3000001:3000001::/home/BUILTIN/server
operators:/bin/false
root at server:/usr/local/samba/etc# wbinfo --uid-info=3000002
NT AUTHORITY\system:*:3000002:3000002::/home/NT AUTHORITY/system:/bin/false
root at server:/usr/local/samba/etc# wbinfo --uid-info=3000003
NT AUTHORITY\authenticated users:*:3000003:3000003::/home/NT
AUTHORITY/authenticated users:/bin/false

samba-tool Group Membership to show SAMDOM\Administrator is inclusive
root at server:/usr/local/samba/etc# samba-tool group listmembers
administrators
Domain Admins
Enterprise Admins
Administrator
root at server:/usr/local/samba/etc# samba-tool group listmembers 'Domain
Admins'
jmirer
Administrator
root at server:/usr/local/samba/etc#

tdbbackup -v results
root at server:/# tdbbackup -v /usr/local/samba/private/secrets.tdb
/usr/local/samba/private/secrets.tdb : 7 records
root at server:/# tdbbackup -v /usr/local/samba/private/schannel_store.tdb
/usr/local/samba/private/schannel_store.tdb : 2 records
root at server:/# tdbbackup -v /usr/local/samba/var/locks/account_policy.tdb
/usr/local/samba/var/locks/account_policy.tdb : 17 records
root at server:/# tdbbackup -v /usr/local/samba/var/locks/registry.tdb
/usr/local/samba/var/locks/registry.tdb : 76 records
root at server:/# tdbbackup -v /usr/local/samba/var/locks/share_info.tdb
/usr/local/samba/var/locks/share_info.tdb : 3 records
root at server:/# tdbbackup -v /usr/local/samba/var/locks/winbindd_cache.tdb
/usr/local/samba/var/locks/winbindd_cache.tdb : 2 records

No duplicate UIDs
root at server:/# ldbsearch -H /usr/local/samba/private/idmap.ldb | grep
xidNumber
xidNumber: 3000014
xidNumber: 3000036
xidNumber: 3000050
xidNumber: 3000040
xidNumber: 3000028
xidNumber: 3000003
xidNumber: 3000021
xidNumber: 3000049
xidNumber: 3000033
xidNumber: 3000023
xidNumber: 3000011
xidNumber: 65534
xidNumber: 3000027
xidNumber: 3000037
xidNumber: 3000020
xidNumber: 3000016
xidNumber: 3000046
xidNumber: 3000045
xidNumber: 3000032
xidNumber: 3000022
xidNumber: 3000038
xidNumber: 3000026
xidNumber: 3000017
xidNumber: 3000012
xidNumber: 0
xidNumber: 3000043
xidNumber: 3000030
xidNumber: 3000018
xidNumber: 3000044
xidNumber: 3000031
xidNumber: 3000009
xidNumber: 3000035
xidNumber: 3000025
xidNumber: 3000000
xidNumber: 3000005
xidNumber: 3000001
xidNumber: 3000008
xidNumber: 3000002
xidNumber: 3000041
xidNumber: 3000015
xidNumber: 3000039
xidNumber: 3000007
xidNumber: 3000006
xidNumber: 3000047
xidNumber: 3000019
xidNumber: 3000013
xidNumber: 100
xidNumber: 3000004
xidNumber: 3000048
xidNumber: 3000042
xidNumber: 3000029
xidNumber: 3000034
xidNumber: 3000024
xidNumber: 3000010

SeDiskOperatorPrivilege applied and active to Domain Admins
root at server:~# net rpc rights list accounts -U 'RADICALLAW\Administrator'
-I server.radicallaw.net
Enter RADICALLAW\Administrator's password:
BUILTIN\Print Operators
SeLoadDriverPrivilege
SeShutdownPrivilege
SeInteractiveLogonRight

BUILTIN\Account Operators
SeInteractiveLogonRight

BUILTIN\Backup Operators
SeBackupPrivilege
SeRestorePrivilege
SeShutdownPrivilege
SeInteractiveLogonRight

BUILTIN\Administrators
SeSecurityPrivilege
SeBackupPrivilege
SeRestorePrivilege
SeSystemtimePrivilege
SeShutdownPrivilege
SeRemoteShutdownPrivilege
SeTakeOwnershipPrivilege
SeDebugPrivilege
SeSystemEnvironmentPrivilege
SeSystemProfilePrivilege
SeProfileSingleProcessPrivilege
SeIncreaseBasePriorityPrivilege
SeLoadDriverPrivilege
SeCreatePagefilePrivilege
SeIncreaseQuotaPrivilege
SeChangeNotifyPrivilege
SeUndockPrivilege
SeManageVolumePrivilege
SeImpersonatePrivilege
SeCreateGlobalPrivilege
SeEnableDelegationPrivilege
SeInteractiveLogonRight
SeNetworkLogonRight
SeRemoteInteractiveLogonRight
SeDiskOperatorPrivilege

BUILTIN\Server Operators
SeBackupPrivilege
SeSystemtimePrivilege
SeRemoteShutdownPrivilege
SeRestorePrivilege
SeShutdownPrivilege
SeInteractiveLogonRight

BUILTIN\Pre-Windows 2000 Compatible Access
SeRemoteInteractiveLogonRight
SeChangeNotifyPrivilege

RADICALLAW\Domain Admins
SeDiskOperatorPrivilege

MY REMEDIATION STEPS
Commands that didn’t work to help me regain sysvol access to update ACLs.
chmod 755 -R sysvol
chmod -R o+rx sysvol
samba-tool sysvolcheck and sysvolreset
samba-tool get --as-sddl /usr/local/samba/var/locks/sysvol/SAMDOM
samba-tool ntacl set (*see below regarding untarred bz2 to compare and
apply previous NTACLs).
* I do have current backups (.bz2) one day PRIOR to the issue. I do see
differences in some of the backup .NTACLs files but only on one object.
I did re-apply the NTACLs on the sysvol/SAMDOM directory and its child
folders with their respective .NTACLs from the backup but didn’t know how
to apply them deeper into the Policies\{7EEV*} folders. The { symbol has me
stumped even using ‘{7EEC…}’format, that command will not apply and gives
me a > symbol.
Tried using samba-tools user create but not working 100%
samba-tool user create nicole2 (I can create a new user and edit too but
cannot logon to Win10PC without error that only Temp Profile is used,
because upon creating the new user and defining
--home-directory=\\\server\\Profiles\\nicole2, the actual directory is not
written to the /Profiles directory.
What next?
I do not want to do a full restore in prod, if I can avoid it. But I do
have an unused off-site dev box I can restore to for comparisons but would
like some step-by-steps on how to do that, and what to search for.

Thank you in advance.
Darren