Evolution of Databases Part I: 20 Years - How Databases Changed While DBAs Vanished

In 1999, every production system had its Albert—a database administrator who typed with two fingers, knew every table and index by heart, and could prevent disasters with a well-timed "no." Today, developers juggle PostgreSQL, Redis, Elasticsearch, and DynamoDB in a single application, often without a database specialist in sight. The numbers tell the story: we've gone from 6 developers per DBA in 2000 to 12:1 (or worse) in 2023, while the number of database systems has exploded from a handful to 426.

This shift reflects fundamental changes in how we build software. The NoSQL revolution, sparked by Google's Bigtable and Amazon's Dynamo papers, shattered the relational monopoly. Facebook gave us Cassandra for inbox search at scale. Graph databases emerged when relationships became as important as the data itself. Redis blurred the line between cache and database. Kafka transformed from message queue to source of truth. Each innovation solved real problems—social graphs needed flexible schemas, IoT devices demanded time-series optimization, real-time features required microsecond access times.

The cloud revolution completed the transformation. AWS RDS, Azure Database, and their NoSQL counterparts didn't just host databases—they absorbed the operational burden that DBAs once managed. Automated backups, failover, patching, and scaling became configuration checkboxes rather than Albert's careful, two-fingered commands. But we've traded his wisdom for automation and choice, and sometimes that trade shows up as 3 a.m. incidents where nobody understands why the writes are backing up. As Tim notes, "Have you ever been humbled by a database?" isn't a trick question—it's a litmus test for whether you've truly lived in production.

Links Main segment

• It's Always the Database - Tim O'Brien on Medium (September 7, 2025) (Note: Specific article link not provided in source material)
• DB-Engines Database Ranking - Track all 426 database systems
• Google Bigtable Paper (OSDI 2006) - The paper that started the NoSQL revolution
• Amazon Dynamo Paper (SOSP 2007) - Introduced eventual consistency
• Jay Kreps: "The Log" - Kafka as database
• Stack Overflow Developer Survey 2023 - Database popularity rankings
• U.S. Bureau of Labor Statistics - Developer and DBA employment data