The cruzdb project began as a completely open-ended exploration of building a MVCC key-value store on top of a distributed shared-log called zlog, and became one of the most interesting and challenging projects I’ve worked. It is able to scale-out across nodes and supports transactions.
This project is a series of deep dives into various components found in the Seastar framework. Seastar is a C++ framework for building high-performance servers that can exploit the full set of capabilities found in modern hardware. However, Seastar represents a departure from the common approach of using a threaded model to build servers, and carries with it a steep learning curve. At Redpanda Data we use Seastar extensively for building our core services, and nearly all the software I’ve written in the past year has been within the context of Seastar.
The Ceph distributed storage system contains a low-level feature called object classes which allow developers to write C++ modules that are loaded directly into the system and implement new behavior. In this project we have embedded the Lua VM into Ceph to allow developers to ship scripts directly into the storage system to implement new behaviors dynamically.