[PATCH] build - use system asn1_compile to cross-compile Samba's bundled Heimdal

[Uri Simchoni]

Hi,

(Taking discussion from https://bugzilla.samba.org/show_bug.cgi?id=14164 
to the list)

Attached is yet another cross-compilation fix which I submitted to 
bugzilla for test by reporter before MR. The fix works for the reporter, 
  but has been Nacked by Andrew, and I'd like to discuss this further here.

Background:
- Heimdal build requires two tools, asn1_compile and compile_et, which 
are binary programs.
- In the native build of Samba with embedded Heimdal, we build them 
first and use them to build the rest of Heimdal.
- This fails on cross-build for obvious reasons.
- One workaround (which I've been using when I was in the embedded Samba 
business) is to build Samba natively, stash the binaries of asn1_compile 
and compile_et somewhere, and set environment vars COMPILE_ET and 
ASN1_COMPILE to point to these binaries.
- A different workaround which is in common use (buildroot, OpenWRT, 
LibreELEC to name just a few) is to *somehow* host-build compile_et and 
asn1_compile and install them in the build's path, then run configure 
with the flag --bundled-libraries='!asn1_compile,!compile_et'. This 
would invoke a configuration test which finds the binaries in the path 
and sets COMPILE_ET / ASN1_COMPILE accordingly. Let's call that "the 
automatic method".
- The "somehow" seems to mostly be building of stand-alone Heimdal 
package. So Samba gets cross-built with asn1_compile that came from 
stand-alone Heimdal, and compile_et which comes from either Heimdal or, 
possibly, another package that produces this binary and has been 
host-built such as e2fsprogs. Both might be overriden by what happens to 
be installed on the build-host, if the distro doesn't take care to put 
its built tools in the path before the system tools.
- commit 8061983d4882f3ba3f12da71443b035d7b672eec broke the automatic 
method, because it invokes the test to find binaries only if using 
system Heimdal.

The attached patch fixes "the automatic method". Andrew Nacked it 
because using some binary that we don't know where it came from is prone 
to incompatibility issues. In fact, it appears that embedded vendors 
routinely use a patch that removes inclusion of <unistd.h> from one of 
Samba's libreplace files, just because of the automatic method, and this 
patch works on 4.11 but breaks master. A better fix would be to get 
Samba's build system to build asn1_compile and compile_et using the host 
compiler.

After this long background, my response to the Nack:

1. Does waf support this mixed-build method? Are there examples / 
pointers? (I can try myself but any pointers would be welcome)
2. Shouldn't we "get things to the way they were" before supplying "the 
perfect fix"? I fear of me running out of time and continuing this at a 
much slower pace.
3. If we nack the fix, would it be correct to say that the current 
situation, where "the automatic method" is invoked in system-heimdal 
build makes no sense, as there's no need for asn1_compile / et_compile 
with system Heimdal?

Thanks,
Uri.
-------------- next part --------------
From b37110d0b380ec6b92a34c36ba39d73ae7590f92 Mon Sep 17 00:00:00 2001
From: Uri Simchoni <uri at samba.org>
Date: Sun, 20 Oct 2019 00:03:14 +0300
Subject: [PATCH] build: find pre-built heimdal build tools in case of embedded
 heimdal

This patch fixes the case of finding asn1_compile and compile_et for
building embedded heimdal, by setting
--bundled-libraries='!asn1_compile,!compile_et' as configure flags.

The Heimdal build tools compile_et and asn1_compile are needed *only*
if we use the embedded heimdal (otherwise we don't build heimdal and
use headers that have been generated by those tools elsewhere).

For cross-compilation with embedded heimdal, it is vital to use host build
tools, and so asn1_compile and compile_et must be supplied and not
built. One way of doing this would be to set the COMPILE_ET and
ASN1_COMPILE env vars to the location of supplied binaries. Another way,
which is more commonly used, is to exclude asn1_compile and compile_et
from bundled packages via the switch
-bundled-libraries='!asn1_compile,!compile_et'. When this is done,
the build script searches the path for those tools and sets the
ASN1_COMPILE and COMPILE_ET vars accordingly. (this is admittedly
kind of a round-about way of doing things but this has become the
de-facto standard amongst embedded distro builders).

In commit 8061983d4882f3ba3f12da71443b035d7b672eec, this process of
finding the binaris has been moved to be carried out only in the
system heimdal case. As explained above, we only need these tools,
and hence the check, in bundled mode.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14164

Signed-off-by: Uri Simchoni <uri at samba.org>
---
 wscript_configure_embedded_heimdal | 11 +++++++++++
 wscript_configure_system_heimdal   | 11 -----------
 2 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/wscript_configure_embedded_heimdal b/wscript_configure_embedded_heimdal
index 8c55ae2a938..4fdae8062c5 100644
--- a/wscript_configure_embedded_heimdal
+++ b/wscript_configure_embedded_heimdal
@@ -1 +1,12 @@
 conf.RECURSE('source4/heimdal_build')
+
+def check_system_heimdal_binary(name):
+    if conf.LIB_MAY_BE_BUNDLED(name):
+        return False
+    if not conf.find_program(name, var=name.upper()):
+        return False
+    conf.define('USING_SYSTEM_%s' % name.upper(), 1)
+    return True
+
+check_system_heimdal_binary("compile_et")
+check_system_heimdal_binary("asn1_compile")
diff --git a/wscript_configure_system_heimdal b/wscript_configure_system_heimdal
index 0ff6dad2f55..f77c177442f 100644
--- a/wscript_configure_system_heimdal
+++ b/wscript_configure_system_heimdal
@@ -36,14 +36,6 @@ def check_system_heimdal_lib(name, functions='', headers='', onlyif=None):
     conf.define('USING_SYSTEM_%s' % name.upper(), 1)
     return True
 
-def check_system_heimdal_binary(name):
-    if conf.LIB_MAY_BE_BUNDLED(name):
-        return False
-    if not conf.find_program(name, var=name.upper()):
-        return False
-    conf.define('USING_SYSTEM_%s' % name.upper(), 1)
-    return True
-
 check_system_heimdal_lib("com_err", "com_right_r com_err", "com_err.h")
 
 if check_system_heimdal_lib("roken", "rk_socket_set_reuseaddr", "roken.h"):
@@ -94,6 +86,3 @@ finally:
 # With the proper checks in place we should be able to build against the system libtommath.
 #if conf.CHECK_BUNDLED_SYSTEM('tommath', checkfunctions='mp_init', headers='tommath.h'):
 #    conf.define('USING_SYSTEM_TOMMATH', 1)
-
-check_system_heimdal_binary("compile_et")
-check_system_heimdal_binary("asn1_compile")
-- 
2.21.0