Problem with configure compiling 64-bit under Solaris 10 -PATCH

[John Center]

Hi,

(Resending to the list) Attached is a file of my changes in what I hope 
is a git-format patch. I changed configure.in against 
origin/v3-3-stable.  It worked for me. Hopefully, this fixes the 64-bit 
problem & I don't have to keep patching configure.  Please let me know 
if it's ok.

HTH

	-John



John Center wrote:
> Hi Volker,
> 
> I'm not sure I know how to do that.  Are there instructions for Samba? 
> I don't have a git repository set up for Samba, yet.
> 
> Thanks.
> 
> 	-John
> 
> 
> Volker Lendecke wrote:
>> On Sun, Mar 15, 2009 at 01:37:32PM -0400, John Center wrote:
>>> There is a problem with configure when trying to compile 64-bit binaries 
>>> on Solaris 10 using Sun Studio 12.  When some of the tests are run, the 
>>> CFLAGS variable is completely replaced with one that contains an option 
>>> for testing.  For example, when testing for compiler warnings, CFLAGS is 
>>> replaced with Werror_FLAGS, which can contain -Werror, -w2 or -errwarn. 
>>>  This fails in 64-bit mode because the appropriate option, "-m64", is 
>>> not included.  Without this flag, conftest fails with a "wrong ELF 
>>> class" error, which configure does not catch.
>>>
>>> The offending lines in configure are often in the form:
>>>
>>> 	old_CFLAGS="$CFLAGS";
>>> 	CFLAGS="$Werror_FLAGS";
>>> 	export CFLAGS;
>>>
>>> where Werror_FLAGS is defined as:  Werror_FLAGS="<errflag>".
>>>
>>> I think Werror_FLAGS should be redefined instead as:
>>>
>>> 	Werror_FLAGS="$CFLAGS <errflag>"
>>>
>>> maintaining the builder's CFLAGS.  This would prevent problems like this 
>>> from occurring in the future.
>> Do you want to provide a git-format-patch style patch?
>>
>> Thanks,
>>
>> Volker
-------------- next part --------------
>From 54b008bff941f7c3714cc23410f79f82d3fc9ac8 Mon Sep 17 00:00:00 2001
From: John Center <john.center at villanova.edu>
Date: Tue, 17 Mar 2009 20:51:17 -0400
Subject: [PATCH] Fixed up places where user's CFLAGS are ignored.

---
 source/configure.in |   16 ++++++++--------
 1 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/source/configure.in b/source/configure.in
index fee3ac1..b5ceb16 100644
--- a/source/configure.in
+++ b/source/configure.in
@@ -286,7 +286,7 @@ AC_CACHE_CHECK([that the C compiler understands -Werror],samba_cv_HAVE_Werror, [
   int main(void)
   {
        return 0;
-  }],[-Werror],[$CPPFLAGS],[$LDFLAGS],
+  }],[$CFLAGS],[-Werror],[$CPPFLAGS],[$LDFLAGS],
   samba_cv_HAVE_Werror=yes,samba_cv_HAVE_Werror=no,samba_cv_HAVE_Werror=cross)])
 if test x"$samba_cv_HAVE_Werror" = x"yes"; then
    Werror_FLAGS="-Werror"
@@ -297,7 +297,7 @@ AC_CACHE_CHECK([that the C compiler understands -w2],samba_cv_HAVE_w2, [
   int main(void)
   {
        return 0;
-  }],[-w2],[$CPPFLAGS],[$LDFLAGS],
+  }],[$CFLAGS],[-w2],[$CPPFLAGS],[$LDFLAGS],
   samba_cv_HAVE_w2=yes,samba_cv_HAVE_w2=no,samba_cv_HAVE_w2=cross)])
 if test x"$samba_cv_HAVE_w2" = x"yes"; then
    Werror_FLAGS="-w2"
@@ -308,7 +308,7 @@ AC_CACHE_CHECK([that the C compiler understands -errwarn],samba_cv_HAVE_errwarn,
    int main(void)
    {
        return 0;
-   }],[-errwarn=%all],[$CPPFLAGS],[$LDFLAGS],
+   }],[$CFLAGS],[-errwarn=%all],[$CPPFLAGS],[$LDFLAGS],
    samba_cv_HAVE_errwarn=yes,samba_cv_HAVE_errwarn=no,samba_cv_HAVE_errwarn=cross)])
 if test x"$samba_cv_HAVE_errwarn" = x"yes"; then
    Werror_FLAGS="-errwarn=%all"
@@ -4015,7 +4015,7 @@ if test x"$with_ads_support" != x"no"; then
                        if (str) free (str);
                        return 0;
                }
-               ],[$Werror_FLAGS],[$CPPFLAGS],[$LDFLAGS],
+               ],[$CFLAGS],[$Werror_FLAGS],[$CPPFLAGS],[$LDFLAGS],
                smb_krb5_cv_enctype_to_string_takes_krb5_context_arg=yes,
                smb_krb5_cv_enctype_to_string_takes_krb5_context_arg=no)])

@@ -4033,7 +4033,7 @@ if test x"$with_ads_support" != x"no"; then
                        krb5_enctype_to_string(1, buf, 256);
                        return 0;
                }
-               ],[$Werror_FLAGS],[$CPPFLAGS],[$LDFLAGS],
+               ],[$CFLAGS],[$Werror_FLAGS],[$CPPFLAGS],[$LDFLAGS],
                smb_krb5_cv_enctype_to_string_takes_size_t_arg=yes,
                smb_krb5_cv_enctype_to_string_takes_size_t_arg=no)])

@@ -4699,7 +4699,7 @@ AC_TRY_RUN_STRICT([
 #define HAVE_QUOTACTL_4A 1
 #define AUTOCONF_TEST 1
 #include "confdefs.h"
-#include "${srcdir-.}/tests/sysquotas.c"],[$Werror_FLAGS],[$CPPFLAGS],[$LDFLAGS],
+#include "${srcdir-.}/tests/sysquotas.c"],[$CFLAGS],[$Werror_FLAGS],[$CPPFLAGS],[$LDFLAGS],
            samba_cv_HAVE_QUOTACTL_4A=yes,samba_cv_HAVE_QUOTACTL_4A=no,samba_cv_HAVE_QUOTACTL_4A=cross)])
 if test x"$samba_cv_HAVE_QUOTACTL_4A" = x"yes"; then
     samba_cv_SYSQUOTA_FOUND=yes;
@@ -4714,7 +4714,7 @@ AC_TRY_RUN_STRICT([
 #define HAVE_QUOTACTL_4B 1
 #define AUTOCONF_TEST 1
 #include "confdefs.h"
-#include "${srcdir-.}/tests/sysquotas.c"],[$Werror_FLAGS],[$CPPFLAGS],[$LDFLAGS],
+#include "${srcdir-.}/tests/sysquotas.c"],[$CFLAGS],[$Werror_FLAGS],[$CPPFLAGS],[$LDFLAGS],
            samba_cv_HAVE_QUOTACTL_4B=yes,samba_cv_HAVE_QUOTACTL_4B=no,samba_cv_HAVE_QUOTACTL_4B=cross)])
 if test x"$samba_cv_HAVE_QUOTACTL_4B" = x"yes"; then
     echo "int quotactl(const char *path, int cmd, int id, char *addr) is not reworked for the new sys_quota api"
@@ -4730,7 +4730,7 @@ AC_TRY_RUN_STRICT([
 #define HAVE_QUOTACTL_3 1
 #define AUTOCONF_TEST 1
 #include "confdefs.h"
-#include "${srcdir-.}/tests/sysquotas.c"],[$Werror_FLAGS],[$CPPFLAGS],[$LDFLAGS],
+#include "${srcdir-.}/tests/sysquotas.c"],[$CFLAGS],[$Werror_FLAGS],[$CPPFLAGS],[$LDFLAGS],
            samba_cv_HAVE_QUOTACTL_3=yes,samba_cv_HAVE_QUOTACTL_3=no,samba_cv_HAVE_QUOTACTL_3=cross)])
 if test x"$samba_cv_HAVE_QUOTACTL_3" = x"yes"; then
     echo "CRAY int quotactl (char *spec, int request, char *arg) is NOT reworked for the sys_quota api"
--
1.6.2