[Samba] Patches for Samba 4.12.0rc3 for OmniOS

[Peter Eriksson]

A little summary of building Samba 4.12.0rc3 on OmniOS:

The following 4 (5) small patches are needed in order to compile and run Samba 4.12.0rc3 on the latest OmniOS r151032p (OpenSolaris/Illumos variant):

1. ARRAY_SIZE is already defined in <sys/sysmacros.h> on OmniOS, put an #ifndef around it in lib/replace/replace.h (can be ignored but causes a gazillion compiler warnings :-):

   https://www.grebo.net/~peter/omnios/samba/4.12/array_size.patch <https://www.grebo.net/~peter/omnios/samba/4.12/array_size.patch>

2. s_addr is a define and causes compilation problems (errors), change the argument to “xs_addr” instead in lib/tsocket/tsocket.h and tsocket_bsd.c:

   https://www.grebo.net/~peter/omnios/samba/4.12/s_addr.patch <https://www.grebo.net/~peter/omnios/samba/4.12/s_addr.patch>

3. The “wscript” build tool generates invalid compiler/linker options (when compiling with Gcc8) - build/link failures:

   https://www.grebo.net/~peter/omnios/samba/4.12/wscript.patch <https://www.grebo.net/~peter/omnios/samba/4.12/wscript.patch>

4. Errors about tdb_mutex_init failing with Device Busy (when using robust mutex:es) at runtime (“net ads info”):

tdb(/var/samba/lock/gencache.tdb): tdb_open_ex: tdb_mutex_init failed for /var/samba/lock/gencache.tdb: Device busy
tdb(/var/samba/lock/gencache.tdb): tdb_open_ex: tdb_mutex_init failed for /var/samba/lock/gencache.tdb: Device busy
tdb(/var/samba/lock/gencache.tdb): tdb_open_ex: tdb_mutex_init failed for /var/samba/lock/gencache.tdb: Device busy

This patch fixes this:

   https://www.grebo.net/~peter/omnios/samba/4.12/mutexattr.patch <https://www.grebo.net/~peter/omnios/samba/4.12/mutexattr.patch>

(Probably an OmniOS bug, but it’s an easy patch in Samba so…)

5. fmemopen() doesn’t exist on OmniOS. This is just a test program and it’s not built by default in rc3 but anyway:

   https://www.grebo.net/~peter/omnios/samba/4.12/fmemopen.patch <https://www.grebo.net/~peter/omnios/samba/4.12/fmemopen.patch>


In order to build it I also compiled and built local versions of:

gmp-6.2.0
gnutls-3.6.12
jansson-2.12
libarchive-3.4.2
libidn2-2.3.0
libtasn1-4.16.0
libunistring-0.9.10
nettle-3.5.1
openldap-2.4.49
p11-kit-0.23.20

I built Samba as a 64 bit binary (gcc -m64). 

With -m32 anything that uses Python fails since the OmniOS Python installation seems to be 64 bit.

- Peter