s3compat system for running s3 components in s4

[tridge at samba.org]

I've been working with Andrew Bartlett on a new approach for more
closely integrating Samba3 and Samba4.

The new approach is described is some detail here:

  http://wiki.samba.org/index.php/Samba4/s3compat

Basically, it builds the Samba3 smbd as a shared library, and uses
hide_symbols from the waf build (which maps to -fvisibility=hidden on
gcc) to prevent symbol conflicts between the s3 and s4 code.

The Samba4 bin/samba binary then links to libs3compatcore.so, and is
able to access s3 functions via a thin compatibility layer (in
s3compat/s3compat.c).

As a first demonstration of what can be done with this, Andrew and I
have added an option:

  s3compat:smbd = True

which enables the Samba3 SMB file server backend in Samba4. It seems
to work for the simple tests that we have tried so far. We were able
to do this without changing any code in Samba3.

There is a lot more that would need to be done to make this useful for
production deployment, especially in the area of the auth code, but it
does provide a useful example of how we might move toward our aim of a
single Samba release that contains components from both Samba4 and
Samba3.

The s3compat:smbd approach is to have Samba4 handle the socket level
accept/fork, and hand over to the s3 code once a connection is
established. It is thus similar to the smb_server/smb_samba3.c code
that Metze did, but avoids the need to have Samba3 installed, and uses
common code (via libraries) for all of the parts of the s3 and s4
codebase that are currently shared. This means that as more code
becomes common that we can see immediate benefits in the functionality
of the combined binary.

The approach of handing over at the socket level is not the eventual
aim. Instead I'd like to see a per-share option for which backend to
use. I think that will be possible, although it will require a lot
more work than what Andrew and I have done over the last couple of
days.

I'd also like to see per-pipe options for which RPC code to use, and
similarly be able to cleanly redirect the printing code to the s3
shared library.

If you want to try it out then please see the wiki page above. It is a
demonstration only at this point, but hopefully it will inspire some
discussions on approaches for increasing the amount of common code
between s3 and s4.

Also note that this whole approach currently only works for systems
where -fvisibility=hidden works, which means gcc at the moment. We may
be able to find the equivalent flags for other compilers, or we could
possibly use linker scripts (authgenerated from the code) to achieve
the same result. Alternatively, we could slowly remove the symbol
conflicts (either through more shared code, or function renames) which
would remove the need for the linker magic in the longer term.

Cheers, Tridge