toyDB

toyDB is a distributed SQL database implemented in Rust, created as a learning project. This database encompasses several advanced features typically found in production systems, providing a valuable resource for those interested in database internals.

Key Components

Raft-based Consensus Engine

At the core of toyDB is a Raft-based distributed consensus engine. This ensures linearizable state machine replication, maintaining consistency across a distributed cluster of nodes.

ACID Transactions

toyDB includes an ACID-compliant transaction engine that employs Multi-Version Concurrency Control (MVCC) to achieve snapshot isolation. This allows multiple transactions to occur concurrently without interference.

Pluggable Storage Engine

The storage layer in toyDB is pluggable, currently supporting both BitCask and in-memory backends. This flexibility allows users to experiment with different storage mechanisms depending on their needs.

Iterator-based Query Engine

The query engine in toyDB is iterator-based and supports heuristic optimization and time-travel capabilities. This enables efficient query processing and the ability to execute historical queries.

SQL Interface

toyDB provides a rich SQL interface that includes support for projections, filters, joins, aggregates, and transactions. This makes it a fully functional SQL database despite its primary role as a learning tool.

Documentation

The toyDB project is well-documented, offering several resources for understanding its architecture and usage:

• Architecture Guide: Detailed insights into the architectural design and implementation of toyDB.
• SQL Examples: Comprehensive examples showcasing the SQL capabilities of toyDB.
• SQL Reference: A detailed reference for toyDB's SQL dialect.
• References: Books and research materials that informed the development of toyDB.

Testing and Benchmarks

toyDB is equipped with extensive tests, ensuring its functionality and reliability:

• Unit Tests: In-code tests for low-level components.
• Integration Tests: Golden master tests for the SQL engine.
• End-to-End Tests: Basic cluster tests to validate overall functionality.

Run cargo test to execute all tests, or view the latest CI results here.

Additionally, toyDB includes a workload benchmarking tool to assess performance under various conditions. Example benchmarks demonstrate the capabilities and performance characteristics of the database.

Debugging

Developers can use Visual Studio Code for debugging toyDB with the included configuration files:

1. Install the CodeLLDB extension.
2. Navigate to the "Run and Debug" tab and select the appropriate debug configuration, such as "Debug unit tests in library 'toydb'" or "Debug executable 'toydb'".

Credits

The logo for toyDB was created by @jonasmerlin.

toyDB is an educational project and not intended for production use. However, it offers an excellent opportunity to delve into the complexities of distributed database systems and learn from a practical implementation.