Current Status

What FerrisDB Is Today

FerrisDB is an educational project where a CRUD developer (human) and an AI (Claude) are learning to build a distributed database from scratch. We’re documenting every step of the journey.

Current state: We have basic storage components but no usable database yet.

Implementation Status

✅ Completed Components

Write-Ahead Log (WAL)

Binary format with 64-byte file headers
CRC32 checksums for data integrity
FileFormat/FileHeader trait system
Comprehensive metrics collection
Zero-copy reads with BytesMutExt
153 tests covering all edge cases
~2,500 lines of tested code

MemTable (Skip List)

Concurrent skip list implementation - Lock-free reads, fine-grained write locks - MVCC support with timestamps - ~450 lines of tested code

SSTable Writer

Basic block format defined - Can write sorted data to disk - Index and footer structures - ~650 lines of tested code

SSTable Reader

Binary search for efficient lookups
Block caching for performance
Footer and index parsing
~400 lines of tested code

🎉 Day 5 Achievements

WAL File Headers

NEW

Implemented 64-byte headers with magic numbers, versioning, and metadata for forward compatibility.

Performance Optimization

NEW

BytesMutExt zero-copy reads achieve 23-33% performance improvement.

Metrics System

NEW

Comprehensive metrics tracking for operations, success rates, and performance monitoring.

🚧 In Progress

Storage Engine Integration - Connecting components together

❌ Not Started

Storage Engine Integration - Components exist separately, not integrated
Compaction - No background merging of SSTables
Server - No network protocol or API
Client Library - No way to connect to a database
Transactions - MVCC timestamps exist but no transaction manager
Distribution - Single-node only, no clustering

You Can…

✅ Today

Clone and explore the codebase
Run tests for individual components
Learn from our blog posts
Follow our tutorials
Read our implementation code

❌ Not Yet

Run a FerrisDB server
Store and retrieve data
Connect with a client
Deploy to production
Benchmark performance

Code Statistics

# Current statistics (as of Day 5)
Total Rust code: 11,306 lines
Total tests: 217
Blog posts: 10 (5 human, 5 Claude)
Development days: 4

# Component breakdown
ferrisdb-storage/src/: ~8,500 lines (including WAL, MemTable, SSTable)
ferrisdb-core/src/: ~200 lines
ferrisdb-tutorials/: ~2,000 lines
ferrisdb-client/src/: ~10 lines (stub)
ferrisdb-server/src/: ~5 lines (stub)

Test Coverage

WAL: 95%+ coverage (153 tests total)
- Unit tests: 108
- Concurrent tests: 5
- Format tests: 22
- Integration tests: 6
- Property tests: 12
MemTable: 85%+ coverage
SSTable Writer: 80%+ coverage
SSTable Reader: 85%+ coverage
Benchmarks: Performance validation for all components
Integration: Basic component integration tests

What’s Next?

We’re learning and building in public. Next steps:

Integrate Components - Create basic storage engine
Add Basic API - Simple get/put operations
Add Compaction - Background SSTable merging
Build Server - Simple key-value API

Follow our blog to see progress in real-time!

Try It Yourself

While you can’t run FerrisDB as a database yet, you can:

# Clone and explore
git clone https://github.com/ferrisdb/ferrisdb
cd ferrisdb

# Run component tests
cargo test -p ferrisdb-storage

# Read the implementation
cat ferrisdb-storage/src/wal/writer.rs

The Honest Truth

We’re at the beginning of a long journey. FerrisDB today is:

📚 Educational - Great for learning, not for production
🏗️ Incomplete - Basic components, no full system
🌱 Growing - Actively developed, check back often
🤝 Collaborative - Watch human-AI partnership in action

Last updated: June 1, 2025