Exploring FerrisDB

Learning Focus

FerrisDB is a codebase for learning, not a database you can run. This guide helps you explore the implementation and understand how databases work internally.

What You’ll Find

Real Components

Working implementations of WAL, MemTable, and SSTable writer

Test Suite

Comprehensive tests showing how each component works

Learning Code

Clear implementations prioritizing understanding over optimization

Prerequisites

Rust 1.81+

Install via rustup.rs

Git

For cloning the repository

Curiosity

Desire to understand how databases work

Getting Started

1. Clone the repository

   git clone https://github.com/ferrisdb/ferrisdb.git
   cd ferrisdb

2. Install Rust toolchain

   rustup toolchain install stable
   rustup component add rustfmt clippy

3. Build the components

   cargo build --all

4. Run the tests

   cargo test --all

Note

First build takes a few minutes. You’re compiling a (partial) database from scratch!

Understanding the Code Structure

ferrisdb/
├── ferrisdb-core/       # Basic types (Key, Value, Error)
├── ferrisdb-storage/    # The interesting stuff!
│   ├── src/
│   │   ├── wal/        # Write-ahead log implementation
│   │   ├── memtable/   # Concurrent skip list
│   │   └── sstable/    # File format for persistence
├── ferrisdb-client/     # Stub only
├── ferrisdb-server/     # Stub only
└── ferrisdb-tutorials/  # Hands-on learning

Exploring Key Components

1. Write-Ahead Log (WAL)

FULLY IMPLEMENTED

The WAL ensures durability - data survives crashes:

# See the implementation
cat ferrisdb-storage/src/wal/writer.rs

# Run WAL-specific tests
cargo test -p ferrisdb-storage wal::

# Trace a write operation
RUST_LOG=debug cargo test test_wal_basic -- --nocapture

What to look for:

• How entries are encoded in binary format
• CRC32 checksum calculation
• fsync calls for durability

2. MemTable (Skip List)

FULLY IMPLEMENTED

A concurrent skip list for in-memory operations:

# Explore the skip list
cat ferrisdb-storage/src/memtable/skip_list.rs

# See concurrent operations in action
cargo test -p ferrisdb-storage skiplist::tests::test_concurrent

# Understand MVCC (multi-version concurrency)
grep -n "timestamp" ferrisdb-storage/src/memtable/skip_list.rs

What to look for:

• Lock-free reading using crossbeam epoch
• How versions are ordered by timestamp
• The clever height selection algorithm

3. SSTable Components

Writer ✅

IMPLEMENTED

# See how data is written to disk
cat ferrisdb-storage/src/sstable/writer.rs

# Test SSTable creation
cargo test -p ferrisdb-storage test_sstable_writer

Creates sorted files with index and footer.

Reader ❌

NOT IMPLEMENTED

# See the skeleton
cat ferrisdb-storage/src/sstable/reader.rs

Structure exists but can’t read files yet.

Learning Exercises

Exercise 1: Trace a Write

# Add debug prints to understand flow
echo 'println!("WAL write: {:?}", entry);' >> ferrisdb-storage/src/wal/writer.rs

# Run and see the output
cargo test test_wal_basic -- --nocapture

# Don't forget to remove your prints!
git checkout ferrisdb-storage/src/wal/writer.rs

Exercise 2: Measure Performance

# Run benchmarks (if any exist)
cargo bench --all

# Time the tests
time cargo test -p ferrisdb-storage

# Check binary size
cargo bloat --release -p ferrisdb-storage

Exercise 3: Find TODOs

# What's left to implement?
grep -r "TODO" ferrisdb-storage/src/

# What's not implemented?
grep -r "unimplemented!" ferrisdb-storage/src/

Common Patterns to Study

Error Handling

Look for Result<T> and ? operator usage throughout the codebase

Concurrency

See how Arc, Mutex, and lock-free structures are used

Binary Encoding

Understand how data is serialized for disk storage

Testing Patterns

Learn from comprehensive test suites for each component

Development Tips

Running Tests Effectively

# Run all tests
cargo test --all

# Run specific test
cargo test test_name

# See println! output
cargo test -- --nocapture

# Run tests in single thread (for debugging)
RUST_TEST_THREADS=1 cargo test

# Use nextest for better output
cargo install cargo-nextest
cargo nextest run --all

Code Exploration Tools

# Generate and view documentation
cargo doc --all --no-deps --open

# See dependency tree
cargo tree -p ferrisdb-storage

# Find a type definition
grep -r "struct SkipList" ferrisdb-storage/src/

What You Can’t Do (Yet)

Caution

Remember, FerrisDB isn’t a working database yet. You cannot: - Start a server - Connect with a client - Store and retrieve data - Run benchmarks against a running system

Contributing Your Learning

Found something interesting? Share it!

1. Document your learning in an issue
2. Fix a TODO you found
3. Add a test for an edge case
4. Improve comments where you got confused

Next Steps

Start Tutorial

Build your own components in our tutorial series

Read the Blog

See how we built these components in our development blog

Pick a Component

Deep dive into WAL, MemTable, or SSTable implementation

Join Discussion

Ask questions and share insights on GitHub

---

Remember: The goal isn’t to use FerrisDB, it’s to understand how databases work by exploring one being built.