smb read/write block size
Christopher R. Hertel
crh at ubiqx.mn.org
Tue Jan 31 23:45:56 GMT 2006
There is some feature in Ethereal, turned off by default, that will
rebuild the original messages and parse them as a whole. I've not yet
used it (though I should) so I don't know exactly where it is.
If I get a chance I'll take a look but I'm booked up this evening so I may
not get to it (and I'll likely forget if it goes more than a day).
Chris -)-----
On Tue, Jan 31, 2006 at 12:15:12PM -0800, Stuart Kendrick wrote:
> hi chris,
>
> ahh, ok, so ... so the NetBios header places its own restriction on size
> ... 2^17 if running over NBT or 2^24 if running naked. ok, i had missed
> this; thanx for noting it.
>
> on another note, do you have any tips for analyzing SMBs which get
> fragmented across more than one IP packet? [i'm speaking from the point
> of view of the sniffer here.] i have one particular SMB client whose
> Session Setup AndX Request gets chunked into two packets: a 1518 byte
> packet and a 1090 byte packet ... Ethereal decodes the first packet just
> fine (most of the packet consists of the Security Blob) ... but Ethereal
> doesn't offer an SMB decode of the second packet, only Ethernet/IP/TCP
> decodes, leaving the TCP payload undecoded. i can see how difficult this
> is to do, from the point of view of the SMB dissector ... and am wondering
> if you've found a work-around?
>
> --sk
>
>
>
> On Sun, 29 Jan 2006, Christopher R. Hertel wrote:
>
> > The absolute upper-limit is something less than 2^24. That's because the
> > maximum size of an SMB message is limited by the 24-bit message size field
> > in the NBT header.
> >
> > The NBT header is used for both NBT and naked transport, so the rule
> > applies regardless. Strictly speaking, under NBT the maximum would be
> > 2^17 bytes.
> >
> > See http://ubiqx.org/cifs/SMB.html#SMB.1.2
> >
> > Of course, those limits do not apply to NT Trans exchanges, which can
> > string together multiple SMB messages to get a single block of data
> > across. NT Trans uses 32-bit sizes.
> >
> > See http://ubiqx.org/cifs/SMB.html#SMB.9.4
> >
> > My brain hurts too.
> >
> > Chris -)-----
> >
> > On Sun, Jan 29, 2006 at 03:50:47PM -0800, Stuart Kendrick wrote:
> > > hi chris,
> > >
> > > ok, i've been chewing on this a while, analyzing Session Setup AndX
> > > Requests from various implementations ... reading your text at
> > > http://ubiqx.org/cifs/SMB.html#SMB.9.2 ... and i don't have a definitive
> > > answer (no surprise there). a key test would be to watch two 64 bit
> > > versions of Windows copy files to one another ... but i haven't done that.
> > >
> > >
> > > however here's what i believe ... at least for the moment:
> > >
> > > -in Session Setup AndX Request, the initiator uses a USHORT to specify
> > > MaxBufferSize "Client's maximum buffer size" (SNIA 1.0) or 'Max Buffer'
> > > (Ethereal) ... ok, the use of a USHORT here places a 64K ceiling on
> > > read/write block sizes.
> > >
> > > -in Read AndX Request, the initiator uses a USHORT to specify MaxCount
> > > "Max number of bytes to return" (SNIA 1.0) or 'Max Count Low' (Ethereal).
> > > and in Read AndX Response, the server uses a USHORT to specify Data Length
> > > "Number of data bytes" (SNIA 1.0) or 'Data Length Low' (Ethereal). ...
> > > ok, the use of a USHORT places a 64K ceiling on read block size.
> > >
> > > -in Write AndX Request, the initiator uses a USHORT to specify DataLength
> > > "Number of data bytes in buffer" (SNIA 1.0) or Data Length Low (Ethereal).
> > > and in Write AndX Response, the server uses a USHORT to specify Count
> > > "Number of bytes written" (SNIA 1.0) or 'Count Low' (Ethereal) ... ok,
> > > the use of a USHORT here places a 64K ceiling on write block size.
> > >
> > >
> > > summary ==> all these USHORTs suggest to me that 64K is the limit on
> > > read/write block size
> > >
> > >
> > > caveat ==>
> > >
> > > my read of Session Setup AndX Request says that if CAP_LARGE_FILE is set
> > > "The client can deal with files having 64 bit offsets" (SNIA v1.0) or
> > > 'Large Files' (Ethereal), then Read AndX Request can employ a ULONG to
> > > specify MaxCountHigh "High 16 bits of MaxCount if CAP_LARGE_READX; else
> > > must be zero" (SNIA 1.0)
> > >
> > > from p.70 of SNIA:
> > > "If CAP_LARGE_READX [sp] was indicated by the server in the negotiate
> > > protocol response, the request's MaxCount field may exceed the negotiated
> > > buffer size if Fid refers to a disk file. The server may arbitrarily
> > > elect to return fewer than MaxCount bytes in response."
> > >
> > > i.e. seems to me that if the two sides agree to think in terms of 64 bit
> > > files, then they can sling TCP streams at each other which exceed 64K in
> > > length. at least on the protocol level, i can see this. on the
> > > implementation level ... i don't understand how i could figure out how
> > > much enough buffer space to set aside to handle the incoming flood. maybe
> > > i would just query my TCP Window Size and allocate an SMB read buffer of
> > > the same size ... that feels gross
> > >
> > > [i have not installed Samba or Windows on a 64 bit machine to see if
> > > either one sets CAP_LARGE_READX and then does what i describe above]
> > >
> > >
> > > anyway, i'm going to go away believing that 32 bit clients can't bust the
> > > 64K barrier on read/write block size ... and therefore must continue to
> > > toodle along with less-than-optimal file transfer performance, both on the
> > > LAN (SMB processing time for the Read AndX/Write AndX commands at 64K
> > > intervals) and on the WAN (can't have more than 64K in flight at a time).
> > >
> > > if you think i'm mistaken, do let me know
> > >
> > > --sk
> > >
> > > stuart kendrick
> > > fhcrc
> > >
> > >
> > >
> > > On Mon, 16 Jan 2006, Christopher R. Hertel wrote:
> > >
> > > > Stuart,
> > > >
> > > > I remember some of the discussion you cite below. There was also a related
> > > > conversation regarding buffer sizes as used by the SMB_ECHO command. The
> > > > notes wound up here: http://ubiqx.org/cifs/SMB.html#SMB.9.2
> > > >
> > > > ...not that any of that helps much with regard to the specific questions
> > > > you're asking.
> > > >
> > > > I would take a look at a capture of the protocol negotiation phase and
> > > > see what numbers are being established at the start of the SMB session.
> > > >
> > > > Hope that's at least a little bit useful.
> > > >
> > > > Chris -)-----
> > > >
> > > > Stuart Kendrick wrote:
> > > > > hi,
> > > > >
> > > > > i'm analyzing read/write performance to various CIFS servers. the
> > > > > largest read block size i'm seeing is 61,440 bytes and the largest write
> > > > > block size is 32,768 bytes
> > > > >
> > > > > SMB Write AndX FID: 0x0009, 61440 bytes at offset 0
> > > > > ...
> > > > > SMB Read AndX Request, FID: 0x000b, 32768 bytes at offset 0
> > > > >
> > > > >
> > > > > what drives this choice? is this something hard-wired into Microsofts
> > > > > implementations of CIFS and the rest of us have just copied it? does
> > > > > this relate to how frequently CIFS wants to commit bytes to disk? is
> > > > > there some way to increase this (registry hacks under Windows, smb.conf
> > > > > fiddles under Samba, etc.)? [i'm not looking for specific registry hacks
> > > > > or smb.conf lines here ... just a conceptual understanding of the magic
> > > > > around '61440' and '32768'.]
> > > > >
> > > > >
> > > > > i've found a thread driven by hertel, with urban, minshall, and others
> > > > > involved, which suggests that a 64K ceiling exists because
> > > > > SMB_COM_WRITE/READ_ANDX is a USHORT, i.e. can only acknowledge 64K bytes
> > > > > ... so since CIFS can't acknowledge more than 64K bytes, therefore CIFS
> > > > > can't send more than 64K bytes.
> > > > >
> > > > > ===============================================================
> > > > > http://archives.neohapsis.com/archives/microsoft/various/cifs/2002-q4/0014.html
> > > > >
> > > > > "Once again, that suggests that you could send a write of more than 64K.
> > > > > The thing is, the SMB_COM_WRITE_ANDX response does *not* have an
> > > > > extension field according to the docs:
> > > > >
> > > > > USHORT Count; Number of bytes written
> > > > >
> > > > > So the server can only report up to 64K written. There's no way to report
> > > > > that more than 64K were written unless one of the Reserved fields is
> > > > > actually the upper 16 bits of the Count."
> > > > > =============================================================
> > > > >
> > > > >
> > > > > would i be correct in pointing to the USHORT definition of
> > > > > SMB_COM_WRITE_ANDX/SMB_COM_READ_ANDX as the proximate cause for a 64K
> > > > > read/write limitation in CIFS?
> > > > >
> > > > >
> > > > > i can see room to improve CIFS performance significantly in the LAN
> > > > > environment, and dramatically in the WAN environment (long, fat pipes),
> > > > > if i can hack my clients and servers to increase their CIFS Read/Write
> > > > > block size above 61440 ==> ergo my queries.
> > > > >
> > > > > --sk
> > > > >
> > > > > stuart kendrick
> > > > > fhcrc
> > > > >
> > > >
> > > >
> > > > --
> > > > "Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X
> > > > Samba Team -- http://www.samba.org/ -)----- Christopher R. Hertel
> > > > jCIFS Team -- http://jcifs.samba.org/ -)----- ubiqx development, uninq.
> > > > ubiqx Team -- http://www.ubiqx.org/ -)----- crh at ubiqx.mn.org
> > > > OnLineBook -- http://ubiqx.org/cifs/ -)----- crh at ubiqx.org
> > > >
> >
> > --
> > "Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X
> > Samba Team -- http://www.samba.org/ -)----- Christopher R. Hertel
> > jCIFS Team -- http://jcifs.samba.org/ -)----- ubiqx development, uninq.
> > ubiqx Team -- http://www.ubiqx.org/ -)----- crh at ubiqx.mn.org
> > OnLineBook -- http://ubiqx.org/cifs/ -)----- crh at ubiqx.org
> >
--
"Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X
Samba Team -- http://www.samba.org/ -)----- Christopher R. Hertel
jCIFS Team -- http://jcifs.samba.org/ -)----- ubiqx development, uninq.
ubiqx Team -- http://www.ubiqx.org/ -)----- crh at ubiqx.mn.org
OnLineBook -- http://ubiqx.org/cifs/ -)----- crh at ubiqx.org
More information about the samba-technical
mailing list