heimdal headers ordering (gssapi/gssapi.h, crypto-headers.h)

[Michael Tokarev]

In the other email, we wrote:

 >> First, the includes.  For example, while compiling
 >> lib/krb5_wrap/gss_samba.c ,
 >> the include-path includes -Ithird_party/heimdal/lib
 >> -Ithird_party/heimdal/lib/gssapi .
 >>
 >> gss_samba.c #includes gss_samba.h which includes
 >> lib/replace/system/gssapi.h,
 >> which - based on HAVE_GSSAPI_GSSAPI_H, includes
 >> <gssapi/gssapi.h>.  The first
 >> include path which has gssapi/gssapi.h is third_party/heimdal/lib,
 >> so we include
 >> third_party/heimdal/lib/gssapi/gssapi.h. But this is a simple
 >> dispatcher file, it merely includes <gssapi/gssapi.h>.
 >> Which, as we know already, is
 >> third_party/heimdal/lib/gssapi/gssapi.h. So we end up
 >>  without all the gssapi definitions altogether.
 >> The correct file to include for <gssapi/gssapi.h>
 >> is third_party/heimdal/lib/gssapi/gssapi/gssapi.h (note the double,
 >> or even
 >> triple, gssapi in there) - so the _second_ -I path should be used
 >> from the
 >> above.  This is quite messy and not really reliable.
 >
 > Again, like symbols, header include paths are dependent on the 'deps='
 > of the subsystems.  So a subsystem is missing a dependency.  We have
 > had reports (to Heimdal, frustratingly) about this, but someone needs
 > to chase it down.
 >
 > See https://bugzilla.samba.org/show_bug.cgi?id=15033

Too bad I didn't know about this when chasing this issue here :)
But it is good anyway to know there's someone else who hit it too ;)
LOL :)

Okay.

The thing here is with include path ordering. And with timing issues,
it seems.

I have logs from two consequtive builds, one of which failed (bad),
and another succeeded (good) without any changes in between (the
same source, the same packages installed from debian since I haven't
run apt update, - everything is the same).

We should list heimdal_build/include *before* heimdal/include in
all cases. Yet the order is random, non-deterministic:

--- good0	2022-03-28 09:59:07.502426113 +0300
+++ bad0	2022-03-28 09:59:01.606521064 +0300
@@ -2,0 +3 @@
+-Ithird_party/heimdal/lib/krb5
@@ -3,0 +5 @@
+-Ithird_party/heimdal/include
@@ -4,0 +7,2 @@
+-Ithird_party/heimdal/lib/roken
+-Ithird_party/heimdal_build/include
@@ -7 +10,0 @@
--Ithird_party/heimdal/lib/krb5
@@ -9 +11,0 @@
--Ithird_party/heimdal_build/include
@@ -12,4 +14,2 @@
--Ithird_party/heimdal/lib/roken
--Ithird_party/heimdal/include
--Ithird_party/heimdal/lib/hcrypto
--Ithird_party/heimdal/lib
+-Ithird_party/heimdal/lib/hcrypto/libtommath
+-Ithird_party/heimdal/lib/wind
@@ -20 +19,0 @@
--Ithird_party/heimdal/lib/hcrypto/libtommath
@@ -22 +21,2 @@
--Ithird_party/heimdal/lib/hdb
+-Ithird_party/heimdal/lib/hcrypto
+-Ithird_party/heimdal/lib
@@ -24 +24 @@
--Ithird_party/heimdal/lib/wind
+-Ithird_party/heimdal/lib/hdb

There, I took one command line (compiling source4/kdc/wdc-samba4.c),
split it into separate lines, saved into two files, filtered
everything related to heimdal, and diffed.  In the bad case,
heimdal/include is moved before heimdal_build/include.
And this break stuff for the reasons already mentioned above.

So we do have obvious ordering instability.

Besides this, we have one more very annoying issue. Namely, it
looks like "next" waf steps always include all previous steps,
complete or in parts.  In particular, `waf install' *always*
rebuilds about 6k files even if nothing at all had changed
in between. While `waf build' builds only about 4k files.

Why it is relevant in this context.

In debian, waf configure step is done with -j1, to avoid
parallel/ordering issues.  However, waf build and waf install
are done with default parallelism. But since `waf install'
re-does some of `waf configure' steps, but this time with
parallelism, this may lead to different results with the
ordering as we see with heimdal includes.

So it looks like we have 2 or 3 separate issues here:
the always-rebuild and install-builds-more-than-build,
and include ordering depending on parallelism.

I'll try to figure out what's going on, but these things
definitely require good understanding of waf internals
and machinery.

The two build logs are available, just in case, at
http://www.corpit.ru/mjt/tmp/samba-heimdal-include-ordering.tar.gz

Thanks,

/mjt