lmdb: limited support for mdb_set_compare and mdb_set_dupsort

[bmatsuo]

Trying to figure out how to support these functions has been a long road that is at least partially documented in #12. But I have finally collected all the information I need about the practical requirements for using the APIs and the inherent limitations/guarantees of cgo. I believe the final pieces of the puzzle were delivered from Ian Lance Taylor on the golang-nuts mailing list.

https://groups.google.com/forum/#!topic/golang-nuts/gvezyWFEAYI

I can proceed with the tentative plan I developed in #12 with work I have started on my branch, bmatsuo/compare-funcs. I will need to write substantial documentation with adequate warnings plastered around the docs. Making use of it in an application is about the trickiest thing I have seen in terms of cgo. Hopefully there will be minimal foot-shooting after the feature's introduction.

Basic requirements for use

Writing a custom comparison function in C is the fastest and simplest way to use custom comparison. The lmdb.h file must be copied from lmdb-go into a local project directory in order to ensure that the C.MDB_val struct is visible (using the same version of the header file is safest). Then a C function with MDB_compare_func signature can be defined. And its pointer can be passed to Txn.SetCompare().

package main

/*
#include "lmdb.h"

static inline int my_comparison_func(const MDB_val *a, const MDB_val *b) {
    // ...
}
*/
import "C"

import "github.com/bmatsuo/lmdb-go/lmdb"

func main() {
    //...
    var dbi lmdb.DBI
    env.Update(func(txn *Txn) (err error) {
        dbi, err = txn.OpenDBI("foodb", lmdb.Create)
        if err != nil {
            return err
        }
        txn.SetCompare(dbi, unsafe.Pointer(&C.my_comparison_func))
    }) 
}

[tbg]

Hi Bryan,

was just poking around the source code for exactly this, glad you've already given it a look.
Would you recommend I give the above a try using your feature branch? I'm not in any type of serious setting (so danger zone is fine as long as it has a chance of working).

cc cockroachdb/cockroach#5220

ps: I'm happy writing the comparator in C. In fact, it's already (mostly) there.

[bmatsuo]

You are welcome to try development against the branch I mentioned in the issue description. It may not always be up to date with master. But it's unlikely to have conflicts so merging master on your own is probably easy and safe enough. Feel free to comment on this branch (edit: thread) if anything like that comes up.

I haven't really been working hard toward this as I don't have a direct use case. But it sounds like you do. I would appreciate any help and feedback you can provide to help get this functionality merged.

There isn't any real solid documentation or good example at the moment. So working with the feature branch may be tricky right now. I'm still not sure how to set things up as cleanly as possible. But writing the comparison function in C should be the easiest thing to do. Additionally, it seems like it's simplest to define the function in a .c/.h file instead of the cgo header comment. But I think you can get things to work either way with some finesse.