From 5cbaf7122714d00b28f061fc16684a7b550b3a68 Mon Sep 17 00:00:00 2001 From: bert hubert Date: Sat, 8 Dec 2018 20:57:39 +0100 Subject: [PATCH] docs --- README.md | 117 ++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 100 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index 0a50c99..abb9c8f 100644 --- a/README.md +++ b/README.md @@ -4,28 +4,32 @@ available for C++17, will support C++ 11 again soon. MIT licensed. [LMDB](http://www.lmdb.tech/doc/index.html) is an outrageously fast key/value store with semantics that make it highly interesting for many -applications. Of specific note, besides speed, is the full support for -transactions and read/write concurrency. LMDB is also famed for its -robustness.. **when used correctly**. +applications. Of specific note, besides speed, is the full support for +transactions and good read/write concurrency. LMDB is also famed for its +robustness.. **when used correctly**. The design of LMDB is elegant and simple, which aids both the performance -and stability. The downside of this elegant design is a plethora of rules -that need to be followed to not break things. In other words, LMDB delivers -great things but only if you use it exactly right. +and stability. The downside of this elegant design is a [nontrivial set of +rules](http://www.lmdb.tech/doc/starting.html) +that [need to be followed](http://www.lmdb.tech/doc/group__mdb.html) to not break things. In other words, LMDB delivers +great things but only if you use it exactly right. This is by [conscious +design](https://twitter.com/hyc_symas/status/1056168832606392320). Among the things to keep in mind when using LMDB natively: * Never open a database file more than once anywhere in your process * Never open more than one transaction within a thread - * .. unless they are all Read Only and have MDB_NOTLS set + * .. unless they are all read-only and have MDB_NOTLS set * When opening a named database, no other threads may do that at the same time * Cursors within RO transactions need freeing, but cursors within RW transactions must not be freed. -Breaking these rules causes no errors, but does lead to silent data -corruption, missing updates, or random crashes. +Breaking these rules causes no immediate errors, but does lead to silent +data corruption, missing updates, or random crashes. Again, this is not an +actual bug in LMDB, it means that LMDB expects you to use it according to +its exact rules. And who are we to disagree? -This LMDB library aims to deliver the full LMDB performance while +The `lmdb-safe` library aims to deliver the full LMDB performance while programmatically making sure the LMDB semantics are adhered to, with very limited overhead. @@ -33,9 +37,23 @@ Most common LMDB functionality is wrapped within this library but the native MDB handles are all available should you want to use functionality we did not (yet) cater for. +# Status +Very early. If using this tiny library, be aware things might change +rapidly. To use, add `lmdb-safe.cc` and `lmdb-safe.hh` to your project. + +# Philosophy +This library tries to not restrict your use of LMDB, nor make it slower, +except on operations that should be rare. The native LMDB handles +(Environment, DBI, Transactions & Cursors) are all available for your direct +use if need be. + +When using `lmdb-safe`, errors "that should never happen" are turned into +exceptions. An error that merely indicates that a key can not be found is +passed on as a regular LMDB error code. + # Example The following example has no overhead compared to native LMDB, but already -exhibits several ways in which lmdb-safe is easier and safer to use: +exhibits several ways in which lmdb-safe automates LMDB constraints: ``` auto env = getMDBEnv("./database", 0, 0600); auto dbi = env->openDB("example", MDB_CREATE); @@ -43,11 +61,11 @@ exhibits several ways in which lmdb-safe is easier and safer to use: ``` The first line requests an LMDB environment for a database hosted in -`./database`. **Within LMDB, it is not allowed to open a database file more -than once**, not even from other threads, not even when using a different LMDB -handle. `getMDBEnv` keeps a registry of LMDB environments, keyed to the -exact inode. If another part of your process requests access to the same -inode, it will get the same environment. +`./database`. **Within LMDB, it is not allowed to open a database file more +than once**, not even from other threads, not even when using a different +LMDB handle. `getMDBEnv` keeps a registry of LMDB environments, keyed to +the exact inode & flags. If another part of your process requests access to +the same inode, it will get the same environment. `MDBEnv` is threadsafe. On the second line, a database is opened within our environment. The semantics of opening or creating a database within LMDB are tricky. With @@ -75,7 +93,7 @@ transaction is aborted automatically. To commit or abort, use `commit()` or LMDB is so fast because it does not copy data unless it really needs to. Memory bandwidth is a huge determinant of performance on modern CPUs. This -wrapper agrees and using modern C++, it is possible to seemlessly use +wrapper agrees, and using modern C++ makes it possible to seemlessly use 'views' on data without copying them. Using these techniques, the call to `txn.put()` sets the "lmdb" string to "great", without making additional copies. @@ -87,3 +105,68 @@ disk. In the final line, we commit the transaction, after which it also becomes available for other threads and processes. +A slightly expanded version of this code can be found in +[basic-example.cc](basic-example.cc). + + +# Cursors, transactions +This example shows how to use cursors and how to mix `lmdb-safe` with direct +calls to mdb. + +``` + auto env = getMDBEnv("./database", 0, 0600); + auto dbi = env->openDB("huge", MDB_CREATE); + auto txn = env->getRWTransaction(); + + unsigned int limit=20000000; +``` + +This is the usual opening sequence. + +``` + auto cursor=txn.getCursor(dbi); + MDB_val key, data; + int count=0; + cout<<"Counting records.. "; cout.flush(); + while(!cursor.get(key, data, count ? MDB_NEXT : MDB_FIRST)) { + count++; + } + cout<<"Have "<