[Samba] Problems building Samba 4.20.2 on Solaris due to Heimdal issues

[Jonathan Cox]

Hello,

I've run into some issues building Samba 4.20.2 due to problems concerning Heimdal. First, I am unable to build with the embed Heimdal due to a variety of errors during compilation of the embedded heimdal. As result, I built a "system" heimdal using the current master branch of heimdal from GitHub. Then I tried to build Samba again using the --with-system-heimdalkrb5, but it immediately fails to build. Since I'm not familiar with the Waf build system, I'm at a loss as to how to resolve the problem. I am building on Solaris 10 1/13 using the GNU toolchain from OpenCSW.org, which is gcc-5.5 on a UltraSPARC IIIi workstation.

If anybody can provide some suggestions, ideas or pointers, I would appreciate it.

(1) Attempt with "System" Heimdal built from master at https://github.com/heimdal/heimdal
I was able to successfully build heimdal on my own. My "system" heimdal is installed in /usr/heimdal, and I set the following environment variables prior to building Samba:

export PATH=/usr/heimdal/bin:/usr/local/bin:/export/home/joncox/data/python3/bin:/opt/csw/bin:$PATH
export PKG_CONFIG_PATH=/usr/heimdal/lib/pkgconfig:/usr/local/lib/pkgconfig:/opt/csw/lib/pkgconfig
export CFLAGS="-O3 -mcpu=ultrasparc3 -I/usr/heimdal/include"
export CXXFLAGS="-O3 -mcpu=ultrasparc3 -I/usr/heimdal/include"
export CPPFLAGS="-I/usr/heimdal/include"
export LDFLAGS="-L/usr/heimdal/lib"
export KRB5_HOME=/usr/heimdal
./configure -v --with-system-heimdalkrb5 --disable-python --without-ad-dc --disable-cups --without-libunwind --without-json
make

During configure, it looks like krb5.h, krb5-config and the rest of the heimdal "system" components that I built are detected. However, when I compile with "make", I immediately get the following message.

'configure' finished successfully (4m53.184s)
bash-3.2$ make -j2
PYTHONHASHSEED=1 WAF_MAKE=1 /export/home/joncox/data/python3/bin/python3 ./buildtools/bin/waf build
Waf: Entering directory `/export/home/joncox/Downloads/samba-samba-4.20.2/bin/default'
        Selected system Heimdal build
Checking project rules ...
ERROR: target samdb.objlist: dependency target com_err not declared

make: *** [Makefile:7: all] Error 1

This seems strange because com_err, gssapi and the other related heimdal components are present and reported by krb5-config. As a wild guess, it seems like --with-system-heimdalkrb5 is mistakenly disabling (or not enabling) the rest of the heimdal components or targets that are required.

bash-3.2$ krb5-config
usage: /usr/heimdal/bin/krb5-config [options] [libraries]
options:   [--help]         show this message
           [--all]          display version, vendor, etc.
           [--version]      display version information
           [--prefix]       display the prefix of Kerberos
           [--exec-prefix]  display the exec_prefix of Kerberos
           [--cflags]       display the CFLAGS required
           [--libs]         display the libraries required to link
           [--deps]         display the dependent libs required
                            for static linking

libraries: krb5             Kerberos 5 applications
           gssapi           GSSAPI applications
           kadm-client      libkadm5 client
           kadm-server      libkadm5 server
           kafs             kafs

(2) Attempt with embedded heimdal
On the other hand, if try to build with the embedded heimdal, it proceeds to build heimdal, but then errors during linking since ld can't resolve "libintl_dgettext". But this is strange since I confirmed that this symbol is present in my libintl.so. I tried setting LDFLAGS=-lintl as well, but this didn't help. Unfortunately, since I'm not familiar with Waf, I really don't know how to force it to link with -lintl, otherwise I presume I could get past this error. WAF doesn't seem to provide particularly helpful output, even if I configure with -v.

How can I force Waf to link with a set of libraries in particular directories?

export CFLAGS="-O3 -mcpu=ultrasparc3"
export CXXFLAGS="-O3 -mcpu=ultrasparc3"
export LDFLAGS="-L/opt/csw/lib -lintl"
./configure -v --disable-python --without-ad-dc --disable-cups --without-libunwind --without-json

Note: the symbol is defined in this library, so WAF must not be linking with it:
bash-3.2$ gnm /opt/csw/lib/libintl.so | grep libintl_dgettext
000020b8 T libintl_dgettext

And the error that immediately results:
bash-3.2$ make
PYTHONHASHSEED=1 WAF_MAKE=1 /export/home/joncox/data/python3/bin/python3 ./buildtools/bin/waf build
Waf: Entering directory `/export/home/joncox/Downloads/samba-samba-4.20.2/bin/default'
        Selected embedded Heimdal build
[324/324] Linking bin/default/third_party/heimdal_build/asn1_compile
Undefined                       first referenced
symbol                             in file
libintl_dgettext                    third_party/heimdal/lib/base/bsearch.c.80.o
Build failed
-> task in 'asn1_compile' failed with exit status 1 (run with -v to display more information)

-Jonathan