[CTDB and GFS] test: scenario and some results

[Ignacio Coupeau]

tridge at samba.org wrote:
> Ignacio,
> 
> A few comments on the ping_pong results for GFS.
> 
> First off, its great that it didn't fail! If it passed all the
> ping_pong tests then thats really good news.
> 
>  > 1,2,3 nodes over the same file without options:
>  > -----------------------------------------------
>  > ./ping_pong  /usr/local/etc/users_1/c 4
>  > 
>  >   1 node:  15897 locks/sec
>  >   2 nodes:  1217 locks/sec
>  >   3 nodes:    33 locks/sec
> 
> The low result for 1 node is perhaps the most surprising. It involves
> no contention, so GFS should be able to serve the lock locally. It is
> not unexpected that it be slower than a local filesystem, but not by a
> factor of 20x.

yes... very strange. I tested it with OCFS and the results are quite 
different.

In local:
/usr/local/etc/ping_pong/ping_pong  /tmp/test 1
   268340 locks/sec

I tested also with the OCFS2 (I know that is not ready for samba) all 
yields about 276195 locks/sec in the test without options and 
num_locks=4, but the tree nodes concurrently seems do no affect more 
than 5% and seems a bit strange.

The test with ping_pong -rw over OCFS2 performs with a lot of
	printf("data increment = %u\n", incr)
lines with incr in the ranges 0..255 and don't let me read the 
locks/sec, so I commented out the line, and yields in locks/sec:
1 node:  ~ 139063
2 nodes: ~   3016	3048
3 nodes: ~    346	320	4

but I don't know if this behavior about data increment is ok for the test.

The -rwm performs a seg fault with OCFS2

in static void ping_pong
...
    p = mmap(NULL, num_locks+1, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
...

mmap2(NULL, 5, PROT_READ|PROT_WRITE, MAP_SHARED, 3, 0) = -1 EINVAL 
(Invalid argument)
brk(0)                                  = 0x8b44000
brk(0x8b65000)                          = 0x8b65000

I don't know if "num_locks+1" fits the "*size_t len" in

void *mmap(void *addr, size_t len, int prot, int flags,
               int fildes, off_t off);


> 
> One of the main design aims of clustered Samba with CTDB is that we
> are now much less dependent on the speed of the above operations than
> we were with previous designs. So the ping_pong test must pass, but if
> its slow then clustered Samba should still be usable, especially with
> "posix locking = no".
> 
> btw, did the "data increment" always show the right value when you ran
> the -rw test? That is an important part of the test as far as
> correctness goes. 

Yes:
at the start of 1st node, prompts 1 line data increment and the 
locks/sec; at the start the 2nd node prompts an other line with "data 
increment = ..."; and at the start of the 3rd node, another line with 
"data increment = ..."
All three nodes writes every second the locks/sec.

-- 
________________________________________________________
Dr. Ignacio Coupeau
Systems and Network Services Director
IT Services
University of Navarra           http://www.unav.edu/
Pamplona, SPAIN                 http://www.unav.es/SI/