Waf rebuilding issue with python3

[Douglas Bagnall]

On 13/12/18 8:09 PM, Volker Lendecke via samba-technical wrote:
> On Thu, Dec 13, 2018 at 01:48:16PM +1300, Andrew Bartlett via samba-technical wrote:
>> On Wed, 2018-12-12 at 18:58 +0100, Stefan Metzmacher wrote:
>>>
>>> Maybe there're some ordering/sorting differences between
>>> python2 and python3. Or is somehow uses a different
>>> hashing algorithm.
>>>
>>> metze
>>
>> Yes, that is it, the per-exec seed for the hash function used to avoid
>> hash attacks on production servers.  Attached is a workaround I've
>> pushed to autobuild while someone finds the rouge hash.
> 
> It's weird to put randomness into the hash determining the file
> content. But that's waf for you...
> 

I can't reproduce it, but buildtools/wafsamba/samba_utils.py around 370
looks suspicious to me:

> # make sure we have md5. some systems don't have it
> try:
>     from hashlib import md5
>     # Even if hashlib.md5 exists, it may be unusable.
>     # Try to use MD5 function. In FIPS mode this will cause an exception
>     # and we'll get to the replacement code
>     foo = md5(b'abcd')

Ignore this bit for a second...

> except:
>     try:
>         import md5
          ^^^^^^^^^^
This will always fail in Python 3 (no md5 module). So we fall back to

>         # repeat the same check here, mere success of import is not enough.
>         # Try to use MD5 function. In FIPS mode this will cause an exception
>         foo = md5.md5(b'abcd')
>     except:
>         Context.SIG_NIL = hash('abcd')
>         class replace_md5(object):
>             def __init__(self):
>                 self.val = None
>             def update(self, val):
>                 self.val = hash((self.val, val))
                             ^^^^
using the builtin randomised hash here.

>             def digest(self):
>                 return str(self.val)
>             def hexdigest(self):
>                 return self.digest().encode('hex')
>         def replace_h_file(filename):
>             f = open(filename, 'rb')
>             m = replace_md5()
>             while (filename):
>                 filename = f.read(100000)
>                 m.update(filename)
>             f.close()
>             return m.digest()
>         Utils.md5 = replace_md5
>         Task.md5 = replace_md5
>         Utils.h_file = replace_h_file
> 

So the question is why/whether the first "from hashlib import md5; md5(b'abcd')" can fail.
I can't see a problem, unless people indeed have FIPS compliant pythons.

(Well, I can see problems in the way the code is silently catching all exceptions,
causing this all this mystery, but that's a slightly different matter).

cheers,
Douglas