Shared memory and IPC

[David Collier-Brown]

Luke Kenneth Casson Leighton wrote:
> > My advice is to avoid MAP_FIXED at all costs.
> absolutely love to.  however, it's a speed improvement of
> a potential two or maybe even three orders of magnitude,
> and so cannot be ignored.

	Ok, the lightbulb over my head just lit up (:-))

	If you're seeing order of magnitude increases
	by going from copy-over-net to mmap(MAP_FIXED),
	then you can go to non-fixed, pay a small cost
	and still make a tremendous profit.

	Consider the following code snippit

	struct {
		int	this;
		long 	that;
		char	*theOther;
		...
		char	placeTheOtherPoints[MAXBUF];
	} thingie;

	base =  mmap(addr, len, PROT_READ|PROT_WRITE, MAP_SHARED, 42, 0)
	...
	myThat = base->that;
	myOther =  base->theOther + base;
	...
	base->theOther = newOther - base;

	The "that" value is just at an offset into the mmap'd
	structure, so getting it's value is trivial: most	
	compilers turn this into a ",base,offset"

	the Other value is trickier: it's a pointer to an address
	in the mmapped area, so you have to add the base before
	dereferencing it and subtract the base before updating it.

	A compiler should be able to turn this into a three-
	instruction sequence, 
		load_immediate R2,base
		add R2, R2; 
		load_long R3,R2,offset

	A good compiler with a register history (e.g., gcc)
	should be able to keep the per-process constant 
	base*2 around in a register, and do the accesses
	in single instructions, or sometims as two
	instructions:

		load_long R3,Rsaved-around,offset
	or
		load R2,twiceBase
		load_long R3,R2,offset

	(That's in a cisc machine language: RISC may do better!)

      Implementations:
      1) If you write the acessors for thingie as functions,
	tell the compiler to inline them, and declare
	a global twiceBase set once, you arguably should get
	either one-instruction or worst-case two-instruction
	accesses to pointers in the structure....

	base =  mmap(addr, len, PROT_READ|PROT_WRITE, MAP_SHARED, 42, 0)
	twiceBase = base *2;

	myThat = base->that;
	myOther =  theOther(base);
	...
	setTheOther(base, newOther);
	
     2) perhaps you can adapt the idl to do this form of
	marshalling/unmarshalling, so as to generate the
	easily optimizable code sequences... I'm way
	too rusty on idl compilers to know if that's
	worthwhile or not!

     3) and I'm sure there are other approaches with O(n)
	behavior that I haven't thought of!

--dave	
-- 
David Collier-Brown,           | Always do right. This will gratify 
Americas Customer Engineering, | some people and astonish the rest.
SunPS Integration Services.    |                      -- Mark Twain
(905) 415-2849                 | davecb at canada.sun.com