Is samba 64bit safe ? (PR#19852)

[David Collier-Brown]

Darren Reed wrote:
> I added -Xc to the command line for SUNWspro/cc and found that
> it also complained about integer/pointer problems for strdup(),
> mktemp() and setting sa_handler at the top of CatchSignal() in
> lib/signal.h.  On Solaris7 (at least), sa_handler is
> void (* _handler)().

	The mismatches are a fruitful area to investigate:
	I looked at the assembler, and, to oversimplify,
	if a pointer return value is misdeclared int && 
	used immediately, it will still be in a register 
	and the compiler will pass it untruncated.

	If it isn't used immediately, it will have it's lower
	half stored and used... and all bets about correctness
	are off!

	Specifically, strcmp(crypt("12345678", "12"),"fred")
	looked like this:

        sethi   %hh(.L90),%l1			-- setup args for
        sethi   %lm(.L90),%g1			-- both strcmp 
        or      %l1,%hm(.L90),%l1		-- and crypt
        or      %g1,%lo(.L90),%g1
        sllx    %l1,32,%l1
        or      %l1,%g1,%l1
        add     %l1,16,%o1

        call    crypt
        mov     %l1,%o0				-- return value is
						-- already in reg
						-- so just move it
        call    strcmp
	add     %l1,24,%o1
        mov     %o0,%l1
        st      %l1,[%fp+2039]			-- store return from
						-- strcmp


> These warnings also cropped up:
> "smbd/nttrans.c", line 2009: warning: integer overflow detected: op "<<"
> which sounds, to me, like missing L's or UL's on the end of numerical
> constants in #define's.  Checking <include/smb.h> showed this to be
> the case - some of the #define's are for (1L<<x) and some are just
> (1<<x).

> Adding in extra prototypes for strdup() and mktemp() to one of the .h
> files appears to have fixed the problem of smbd core dumping on
> Solaris7 when compiled as a 64bit binary (using -xarch=v9) with
> SUNWspro 5.0's cc. 

	Excellent!  Relieves my mind.
	Of course, this doesn't explain why crypt failed in
	my case, but that's a question for the Sun cc folks, 
	not us.

--dave
-- 
David Collier-Brown,  | Always do right. This will gratify some people
185 Ellerslie Ave.,   | and astonish the rest.        -- Mark Twain
Willowdale, Ontario   | http://java.science.yorku.ca/~davecb
Work: (905) 415-2849 Home: (416) 223-8968 Email: davecb at canada.sun.com