[PATCH] build - use system asn1_compile to cross-compile Samba's bundled Heimdal

[Andrew Bartlett]

On Sun, 2019-10-20 at 09:37 +0300, Uri Simchoni wrote:
> Hi,
> 
> (Taking discussion from https://bugzilla.samba.org/show_bug.cgi?id=14164 
> to the list)
> 
> Attached is yet another cross-compilation fix which I submitted to 
> bugzilla for test by reporter before MR. The fix works for the reporter, 
>   but has been Nacked by Andrew, and I'd like to discuss this further here.
> 
> Background:
> - Heimdal build requires two tools, asn1_compile and compile_et, which 
> are binary programs.
> - In the native build of Samba with embedded Heimdal, we build them 
> first and use them to build the rest of Heimdal.
> - This fails on cross-build for obvious reasons.
> - One workaround (which I've been using when I was in the embedded Samba 
> business) is to build Samba natively, stash the binaries of asn1_compile 
> and compile_et somewhere, and set environment vars COMPILE_ET and 
> ASN1_COMPILE to point to these binaries.
> - A different workaround which is in common use (buildroot, OpenWRT, 
> LibreELEC to name just a few) is to *somehow* host-build compile_et and 
> asn1_compile and install them in the build's path, then run configure 
> with the flag --bundled-libraries='!asn1_compile,!compile_et'. This 
> would invoke a configuration test which finds the binaries in the path 
> and sets COMPILE_ET / ASN1_COMPILE accordingly. Let's call that "the 
> automatic method".
> - The "somehow" seems to mostly be building of stand-alone Heimdal 
> package. So Samba gets cross-built with asn1_compile that came from 
> stand-alone Heimdal, and compile_et which comes from either Heimdal or, 
> possibly, another package that produces this binary and has been 
> host-built such as e2fsprogs. Both might be overriden by what happens to 
> be installed on the build-host, if the distro doesn't take care to put 
> its built tools in the path before the system tools.
> - commit 8061983d4882f3ba3f12da71443b035d7b672eec broke the automatic 
> method, because it invokes the test to find binaries only if using 
> system Heimdal.
> 
> The attached patch fixes "the automatic method". Andrew Nacked it 
> because using some binary that we don't know where it came from is prone 
> to incompatibility issues. In fact, it appears that embedded vendors 
> routinely use a patch that removes inclusion of <unistd.h> from one of 
> Samba's libreplace files, just because of the automatic method, and this 
> patch works on 4.11 but breaks master. A better fix would be to get 
> Samba's build system to build asn1_compile and compile_et using the host 
> compiler.
> 
> After this long background, my response to the Nack:
> 
> 1. Does waf support this mixed-build method? Are there examples / 
> pointers? (I can try myself but any pointers would be welcome)
> 2. Shouldn't we "get things to the way they were" before supplying "the 
> perfect fix"? I fear of me running out of time and continuing this at a 
> much slower pace.
> 3. If we nack the fix, would it be correct to say that the current 
> situation, where "the automatic method" is invoked in system-heimdal 
> build makes no sense, as there's no need for asn1_compile / et_compile 
> with system Heimdal?

I think using that for cross-compiled file servers the system MIT
Kerberos should be the first preference, not our embedded Heimdal.

The second preference for cross-compiled operation should be a system
Heimdal.

Only if those are not possible, then we should cross-compile Samba's
internal Heimdal.  We should take the patch to fix waf to have cross-
compilation work in principle. 

Samba's waf is meant to be able to prepare host binaries using
use_hostcc=yes on the targets, but that may well have bit-rotted. 

The problem I have with the manual/automatic finding of the host
asn1compile and compile_et binaries is that clearly these are being
mixed up, and developers with little time but the best of intentions
are even hacking around that.  Samba is too complex to have things only
work with time and care, it should just work or not be available at
all.

I say clearly mixed up because:
 - The Heimdal 7.7.0 compile_et and Samba master are almost identical
 - identical compilers would produce identical output, regardless of
architecture. 
 - They would certainly not require different header files.
 - The complained-about output comes from MIT's old compile_et.

Substituting 'just a bit' of our Heimdal build is just asking for
trouble, in an already difficult area.

I understand you are running out of time, my suggestion is to lock in
what we can agree on for now until someone can either make a fool-proof 
fully automatic recipe, one way or the other.

Sorry,

Andrew Bartlett
-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba