Inspired by CMU's Database Course

A random spark of curiosity led me to explore how to create my own database. This quest brought me to the "Database Systems" course taught by Professor Andy Pavlo at Carnegie Mellon University (CMU). Intrigued, I decided to dive in.

Why the Sudden Interest in Databases?

The allure of emerging databases like DuckDB, Neon, and TiDB has been captivating. After reading "Designing Data-Intensive Applications" with Ken and others at London Tech Talk, I thought it would be cool to contribute to this field. It’s a kind of admiration, almost like a dream. Plus, deepening my database knowledge could become a significant strength in my current job. In fact, excelling in SRE combined with something unique seems essential for my career growth.

Initially, I visited the GitHub pages of databases like DuckDB, hoping to contribute, but realized I had no idea where to start. After all, I had never built a database before. While pondering my next steps, I stumbled upon a Reddit thread where someone recommended Professor Andy Pavlo's course. This motivated me to watch his lectures and try building a database myself. Also, seeing my former colleague Teppei thrive at Georgia Tech's graduate program nudged me towards taking university courses seriously.

Choosing the Right Course

I originally considered taking the Advanced Database Systems (15-721) course focused on building an OLAP database with Rust. However, it seemed too advanced, so I opted for the introductory version, Database Systems (15-445/645).

Out of countless course options, I chose this one because the professor seemed charismatic and the course far from boring. A glance at YouTube shows it’s not your typical university class; it even features a dedicated DJ who plays music at the beginning and end. This course is highly popular at CMU, making it a rare opportunity to access such valuable content online. The latest lectures are uploaded each semester, providing a sense of real-time learning. I aim to complete the assignments to experience the course as a real student would.

Progress and Learnings So Far

My goal for this year is to finish the intro course and start building a database by progressing to the advanced level. Although balancing this with parenting means progress is slow, I started watching the videos in March and have just finished the third lecture of the intro course.

Here’s a brief overview of what I’ve covered:

Lecture 1: Skipped halfway due to poor audio quality.
Lecture 2: Focused on SQL syntax, revisiting many known concepts and learning new ones I hadn’t fully mastered.
Lecture 3: Discussed Storage Part 1, including why DBMS should not rely on the memory OS, how database records (tuples) are stored in pages, and the execution of compaction in slotted pages (similar to Postgres' VACUUM). The course uniquely explains SQL dialect differences and behaviors across databases like SQLite, Postgres, Oracle, and DuckDB, making it fascinating.

Looking Ahead

Though it’s a long journey ahead, my dream is to advance and eventually implement a database in Rust. I’m taking it one step at a time.