triple allocation of large buffers.

[Andrew Bartlett]

On Wed, 2008-01-16 at 17:32 +0000, Amin Azez wrote:
> I'm trying to avoid double and triple allocation of large buffers.
> 
> Buffers for read-requests are allocated before vfs_cifs sees the
> request, pointers to these buffers are passed in smb_read structures
> when the request is proxied. I think I remember that the buffer provided
> by the caller is an offset in an already allocated packet response, in
> order to save copying great buffers of memory all over the place.
> 
> vfs_proxy now uses rpc for passing proxied read requests in order to
> easily support various compression types, and I'm wondering if this is a
> bad idea...

That certainly is the down-side to DCE/RPC.  Lots of things are done for
you, but they certainly may not be hand-optimised.  

> How many copies of the RPC large data block will be allocated at once?
> 1. The caller to vfs_proxy pre-allocates a buffer as part of the
> response packet
> 2. The rpc..recv will allocate some when it receives the RPC response.
> 3. Will the rpc struct pointer merely point to an offset in the
> DATA_BLOB or will a new buffer be allocated to hold what is also in the
> data blob?
> 
> I guess I will than have to memcpy from the rpc struct's pointer to the
> req's allocated buffer.
> 
> The whole thing is horrible! If NDR has to unpack to a 3rd buffer then
> there must be some way to hint to unpack it into the existing buffer as
> part of the response packet.

Once it produces a DATA_BLOB in the unmarshaled output, you are free to
put that pointer into the structure to be send to the client, if it
happens to fit.  Otherwise, I suspect a memcpy may be required. 

> But maybe all this just happens transparently anyway with normal reads
> in vfs_cifs, maybe it also has to copy from the received proxy response
> to the outgoing response?

Not having looked at the code, I wonder if vfs_cifs gets away with a lot
because the structures and packets being passed along are identical, so
much less munging is involved.

I'm sorry you seem to be getting the run-around.  This is a new area,
and trying the different options is I hope worthwhile, in the long term.
(If frustrating in the short term). 

Perhaps I could make another suggestion:  Put the data in normal CIFS
packets, but marshal your header with NDR?

Andrew Bartlett

-- 
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org
Samba Developer, Red Hat Inc.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/samba-technical/attachments/20080117/e4c5091b/attachment.bin