Storage backend using RocksDB database

[TimonTerBraak]

The RocksDB library provides a persistent key value store. Keys and values are arbitrary byte arrays. The keys are ordered within the key value store according to a user-specified comparator function.

Implementation is in C++, there are rust and python binding available. The database is stored in a single directory (not single-file).

https://github.com/facebook/rocksdb/blob/gh-pages-old/intro.pdf?raw=true
https://github.com/facebook/rocksdb/wiki

Python bindings for rocksdb:
https://python-rocksdb.readthedocs.io/en/latest/index.html

Rust bindings for rocksdb:
https://github.com/rust-rocksdb/rust-rocksdb

[windelbouwman]

This looks nice!

Having a database backed datastore is very good. Also the key-value pair is handy. It would be nice to be able to change the database, so the key-value API would be a good abstraction for the database.

[TimonTerBraak]

I am writing the layer on top on the Python rocksDB API to be able to read/write the nodes of the btree. That seems to work fine. However, the data structure manipulations are not implemented. That should remain in the btree class.

API is then something like this:

key = addNode(self, left = -1, right = -1, aggregation = [])
key = addLeaf(self, samples)
get(self, key)