talloc_realloc

[Luke Kenneth Casson Leighton]

On Mon, 5 Mar 2001, Jeremy Allison wrote:

> On Tue, Mar 06, 2001 at 12:31:13AM +1100, Luke Kenneth Casson Leighton wrote:
> 
> > when i described to dr andrew tridgell that i was adding a talloc_realloc,
> > i was informed that this makes the purpose of talloc - trivial alloc -
> > non-trivial.  therefore, the function is unacceptable.
> > 
> > now, jeremy adds talloc_realloc.
> > 
> > this implies that the rejection of the addition of this function was not
> > for technical reasons.
> 
> I know I shouldn't answer this but......
> 
> *I* added talloc_realloc to the 2.2 tree without consultation with
> Andrew in order to try and fix the same problem you describe.

i know.
 
> I eventually got it working (it was non-trivial) and after much
> testing (until 1am :-) discovered that when enumerating a 10,000
> printer job queue list it *still* used more than 20mb of virtual
> memory more than the less elegent realloc code (which I left working
> in head).


> Dishartened, I went home. That was over the weekend. As I'm working
> from home today on a white paper I didn't get chance to revert the 
> changes to 2.2 back to the code used in HEAD. I will be doing so
> tomorrow and that implementation of talloc_realloc will be history.
> 
> So yes it *was* for technical reasons, you just need to be able to
> do the tests and determine this for yourself. Now the question remains,
> did *you* do those tests and remove it in your code, or did you just
> leave it in for the case you tested assuming it was ok for all cases ?

i used trealloc() only for the marshalling buffer memory, as using talloc
was utterly hammering memory.

so it was a very specific and limited usage: a tool - an optimisation -
for a job.

so, it was there for one specific case - not all cases.

if you are using talloc_realloc for general memory allocation, including
structures used by the function calls, then well yeah, you're going to get
problems!

ESPECIALLY with the printing code.  remember, the functions go like this:

client:
spoolss_get_printer_data(1024, void*buffer, size_t **size_needed)

server: error_buff_to_small, size_needed = 1024000.

client:
spoolss_get_printer_data(1024000, void *buffer)


but the client-side code in nt is so STUPID in places that it continues to
use the buffer size 1024000 even though the buffer _actuall_ required for
a particular structure may be only 1 byte.



have you considered using more than one TALLOC_CTX?

one for marshalling, one for unmarshalling, one for memory returned to
client-side usage, one for memory returned for server-side usage?


 
> This is why Andrew is the head of the Team, 'cos he can see these
> things without the pain I have to go through to see them :-) :-).

andrew has specific technical skills in specific areas.