[SCM] CTDB repository - branch delete-empty-records created - ctdb-1.9.1-266-gb89961e

[Ronnie Sahlberg]

The branch, delete-empty-records has been created
        at  b89961e6a4eef6aad59e018fe8ab2f250da67ce1 (commit)

- Log -----------------------------------------------------------------
commit b89961e6a4eef6aad59e018fe8ab2f250da67ce1
Merge: b8847a665d2fc56d8f8c8a7d5110d7aafa85bb47 6936f2bcddfa7372d428fc6f1dec4dc741fe269c
Author: Ronnie Sahlberg <sahlberg at lenovo-laptop.(none)>
Date:   Mon Dec 6 16:41:40 2010 +1100

    Merge branch 'delete-empty-records' of 10.1.1.27:/shared/ctdb/ctdb-git into delete-empty-records

commit 6936f2bcddfa7372d428fc6f1dec4dc741fe269c
Author: Ronnie Sahlberg <ronniesahlberg at gmail.com>
Date:   Mon Dec 6 16:09:38 2010 +1100

    Add a new header flag for "migrated with data" and set this to 1
    when we migrate a non-empty record onto the node
    or a non-empty record off the node
    
    When we migrate a record back to the lmaster and yield the dmaster role,
    inspect this flag if if it is still not set, we can delete the record from
    the local database as soon as we have migrated it back to the lmaster.

commit c95c3b007cd5b93de36f616c4c68221c1dfae59b
Author: Ronnie Sahlberg <ronniesahlberg at gmail.com>
Date:   Mon Dec 6 16:07:55 2010 +1100

    add new command line functions
    ctdb readkey <dbid> <key>
    ctdb writekey <dbid> <key> <value>
    
    these are mainly intended for debugging of databases and dmaster migration issues

commit db4029253597d8006e5797b70f8a52c3e4f19470
Author: Ronnie Sahlberg <ronniesahlberg at gmail.com>
Date:   Mon Dec 6 16:06:20 2010 +1100

    add a new ctdb_ltdb function to delete a record in a normal database

commit 3036dbc448ae99672bf390ff9495159838037e8c
Author: Michael Adam <obnox at samba.org>
Date:   Fri Dec 3 15:21:51 2010 +0100

    server: when we migrate off a record with data, set the MIGRATED_WITH_DATA flag

commit 09f3db8f5231aba04844335ad467e3b12f062d4f
Author: Ronnie Sahlberg <ronniesahlberg at gmail.com>
Date:   Mon Dec 6 13:08:53 2010 +1100

    Add two new flags for the ltdb header.
    One of which signals that the record has never been migrated to/from a node
    while containing data.
    This property "has never been migrated while non-zero" is important later
    to provide heuristics on which records we might be able to purge
    from the tdb files cheaply, i.e. without having to rely on the full-blown
    database vacuum.
    
    These records are belived to be very common and the pattern would look like
    this :
    1, no record exists at all.
    2, client opens a file
    3, samba requests the record for this file
    4, an empty record is created on the LMASTER
    5, the empty record is migrated to the DMASTER
    6, samba writes a <sharemode> to the record locally and the record grows
    7, client finishes working the file and closes the file
    8, samba removes the sharemode and the record becomes empty again.
    9, much later : vacuuming will delete the record
    
    At stage 8, since the record has never been migrated onto a node wile being
    non-zero it would be safe, and much more efficient to just delete the record
    completely from the database and hand it back to the LMASTER.
    
    The flags occupy the same uint32_t as was previously used for laccessor/lacount
    in the header. For now, make sure the flags only define/use the top 16 bits
    of this field so that we are sure we dont collide with bits set to one
    from previous generations of the ctdb cluster database prior to this
    change in semantics of this word.
    
    This is a rework of Michaels patch :
    commit 2af1a47cbe1a608496c8caf3eb0c990eb7259a0d
    Author: Michael Adam <obnox at samba.org>
    Date:   Tue Nov 30 17:00:54 2010 +0100
    
        add a DEFAULT record flag and a MIGRATED_WITH_DATA record flag.

commit 7aa00ef043bda5bc1f26529d176d23f7f8e6c861
Author: Ronnie Sahlberg <ronniesahlberg at gmail.com>
Date:   Mon Dec 6 13:04:44 2010 +1100

    change one of the reserved words in the ctdb ltdb header to be a flags field
    for now, try avoiding using bits in the low16 bits as flags since this may
    collide with laccessor/lacount values from previous versions of the cluster
    databases

commit 3bda77185b0882ea43c2316de897a94a51e6390e
Author: Ronnie Sahlberg <ronniesahlberg at gmail.com>
Date:   Mon Nov 29 13:07:59 2010 +1100

    Remove LACOUNT and LACCESSOR and migrate the records immediately.
    
    This concept didnt work out and it is really just as expensive as a full migration
    anyway, without the benefit of caching the data for subsequence accesses.
    
    Now, migrate the records immediately on first access.
    This will be combined with a "cheap vacuum-lite" for special empty records to
    prevent growth of databases.
    
    Later extensions to mimic read-only behaviour of records will include proper shared read-only locking of database records, making the laccessor/lacount read-only access to the data obsolete anyway.
    
    By removing this special case and handling of lacount laccessor makes the codapath where shared read-only locking will be be implemented simpler, and frees up space in the ctdb_ltdb header for use by vacuuming flags as well as read-only locking flags.

-----------------------------------------------------------------------


-- 
CTDB repository