NTVFS to S3FS Migration

[brendan powers]

Since the release of Alpha21, I've been looking into implementing S3FS
in existing NTVFS installations. For the most part, it's fairly
simple. Just added the needed entries to the existing smb.conf file.
There is one sticky point however, and that is upgrading to the new
permission format. In the past, NTVFS has simply stored the security
descriptor in the seciry.NTACL xattr. Now, with the introduction of
S3FS this format has changed to the security descriptor, plus a hash
of the POSIX permissions. As I understand it, if the hash in the xattr
does not match the actual POSIX permissions, the NT descriptor is
thrown out, and then re-built from using the POSIX permissions as a
starting point.

So, at the moment, I'm trying to figure out how to upgrade to S3FS and
preserve any existing NTVFS permissions. As well as migrating new
installations, Resara  Server (the product I work on that's currently
using NTVFS) edits the security attributes in security.NTACL directly.
So I also need a way to edit permissions in S3FS on an ongoing basis.
As I see it, here are my options.

1) Ignore the current ACL format, and just write the security
descriptor directly to the xattr as I have been. This probably isn't
going to work, as I understand it, Samba will ignore this ACL, and use
the POSIX attributes instead. Even if this isn't the case, and Samba
respects the xattr, the POSIX permissions will now be out of date.
2) Update the security descriptor in the xattr using the new hash
format, then manually sync the POSIX permissions, and generate the
hash. This is a fairly difficult option, as I would need to properly
map the POSIX permissions from the NT ACL, get the UIDs from winbind,
and correctly generate the hash. This seems fairly error prone to me,
as the POSIX mappings are a complicated process. I'd need to match the
behavior of Samba exactly as to not introduce subtle bugs.
3) Update the security descriptor in the xattr, and somehow prompt
samba to update the POSIX permissions and the new hash.  This is
probably my preferred method, as it let's me do things mostly the same
way I've been doing them. However, I'm not sure Samba currently
exposes the right methods for me to do this.
4) Use SMB to set the NT ACL as a client would. This has the advantage
of using the same code-path that most other client's use, and doesn't
require any code changes or new APIs. However, this would only allow
changes to files currently shared by the file server. This is somewhat
inconvenient, but could probably be worked around.

So, that's my thoughts on the subject at the moment. This email
started out discussing the upgrade process from NTFS to S3FS and ended
up being more about how an external program can edit NT permissions.
However, I think the two issues are closely related.