3.0.11 with AIX 5.3, unable to claim a session

Fri Mar 18 21:30:00 GMT 2005

I've found that if I force the tdb_open in session_init and
claim_connection, it's working (but this modification is not acceptable)

My TDB_CONTEXT is screwed between reply_sesssetup_and_X() and
session_claim(). More exactly, in getgrouplist_internals() right after
calling initgroups()

AIX 5.3 (Not ok)
stopped in getgrouplist_internals at line 69 in file "lib/system_smbd.c"
($t1)
   69           if (initgroups(user, gid) != 0) {
(dbx) print session.tdb->name
"/usr/local/samba/var/locks/sessionid.tdb"
(dbx) step
stopped in getgrouplist_internals at line 70 in file "lib/system_smbd.c"
($t1)
   70                   DEBUG(0, ("getgrouplist_internals: initgroups()
failed!\n"));
(dbx) print session.tdb->name
(nil)

AIX 4.3.3 (Ok)
[1] stopped in initgroups at 0xd027f534
0xd027f534 (initgroups)    bf01ffe0       stmw   r24,-32(r1)
(dbx) print session.tdb->name
"/usr/local/samba/var/locks/sessionid.tdb"
(dbx) step
stopped in getgrouplist_internals at line 70 in file "lib/system_smbd.c"
   70                   DEBUG(0, ("getgrouplist_internals: initgroups()
failed!\n"));
(dbx) print session.tdb->name
"/usr/local/samba/var/locks/sessionid.tdb"

if this invoke something to anyone, I would like to know

thanx

Yannick Bergeron
yaberger at ca.ibm.com

----- Forwarded by Yannick Y Bergeron/Bromont/IBM on 2005-03-18 16:16 -----

             Yannick Y                                                     
             Bergeron/Bromont/                                             
             IBM                                                        To 
                                       Yannick Y                           
             2005-03-16 14:36          Bergeron/Bromont/IBM at BROMONT        
                                                                        cc 

                                                                   Subject 
                                       3.0.11 with AIX 5.3, unable to      
                                       claim a session                     

Hi,

I'm trying to build samba-3.0.11 on AIX 5.3 ML 1. The compilation is
working fine. I can start smbd server without any problem, but I'm unable
to connect a share to it (the authentication is working). Most of the time,
it's coredumping.
I'm not experiencing this problem on AIX 4.3.3 and AIX 5.2. I've tried with
those compiler: ibmcxx 3.6.6 PTF8 and IBM XL C for Enterprise for AIX 7.0
PTF 1

The problem seem to be with the TDB_CONTEXT tdb in smbd/session.c. It is
initialized when we start the server with the session_init() function and
it's useable (I've tried tdb->hash_fn(&key) right after the TDB_CONTEXT is
created).
When a first session is requested, it's unable to use the TDB_CONTEXT that
was previously created by the server. It's crashing when calling a method
of tdb (tbd->hash_fn(&key) in this case, when tdb_store() is called from
session_claim()).

Could this be because the memory declared for the tdb variable in the smbd
server could not be use by the child process?

Here is the backtrace of a core file (from dbx and from gdb)

==> dbx /usr/local/samba/sbin/smbd core
Type 'help' for help.
[using memory image in core]
reading symbolic information ...

Illegal instruction (illegal opcode) in . at 0x0 ($t1)
warning: Unable to access address 0x0 from core
(dbx) where
<.() at 0x0
tdb_store(0x2ff21848, 0x2ff21340, 0xd, 0x2ff21440, 0x610, 0x1), line 1471
in "tdb.c"
session_claim(??), line 143 in "session.c"
register_vuid(0x30189318, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x2ff2211c), line
252 in "password.c"
reply_sesssetup_and_X(??, ??, ??, ??, ??), line 915 in "sesssetup.c"
switch_message(??, ??, ??, ??, ??), line 968 in "process.c"
construct_reply(??, ??, ??, ??), line 998 in "process.c"
process_smb(??, ??), line 1098 in "process.c"
unnamed block in smbd_process(), line 1558 in "process.c"
smbd_process(), line 1558 in "process.c"
main(argc = 0, argv = (nil)), line 951 in "server.c"

==> gdb /usr/local/samba/sbin/smbd core
GNU gdb 6.2.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you
are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "powerpc-ibm-aix5.1.0.0"...
Core was generated by `smbd'.
Program terminated with signal 4, Illegal instruction.
#0  0x00000000 in ?? ()
(gdb) bt
#0  0x00000000 in ?? ()
#1  0x100606d0 in tdb_store (tdb=0x2ff21848, key={dptr = 0x2ff21340
"ID/40216/100", dsize = 13}, dbuf={dptr = 0x2ff21440 "", dsize = 1552},
flag=1) at tdb/tdb.c:1471
#2  0x101152ac in session_claim (vuser=0x0) at smbd/session.c:143
#3  0x100abc98 in register_vuid (server_info=0x30189318, session_key={data
= 0x0, length = 0, free = @0x0: 0x1}, response_blob={data = 0x0, length =
0, free = @0x0: 0x1}, smb_name=0x2ff2211c "yaberge2") at
smbd/password.c:252
#4  0x102126cc in reply_sesssetup_and_X (conn=0x0, inbuf=0x0,
outbuf=0x42360484 <Address 0x42360484 out of bounds>, length=806645800,
bufsize=0) at smbd/sesssetup.c:915
#5  0x100320dc in switch_message (type=0, inbuf=0x0, outbuf=0x0, size=0,
bufsize=0) at smbd/process.c:968
#6  0x10031b28 in construct_reply (inbuf=0x300104f0
"receive_message_or_smb", outbuf=0x3000ffbc "smbd/process.c",
size=804399040, bufsize=-559038737) at smbd/process.c:998
#7  0x100339e0 in process_smb (inbuf=0x1007252c "\200A", outbuf=0x0) at
smbd/process.c:1098
#8  0x1003327c in smbd_process () at smbd/process.c:1558
#9  0x10000d24 in main (argc=0, argv=0x0) at smbd/server.c:951

Yannick Bergeron
yaberger at ca.ibm.com