Why PostgreSQL Won — and What the Household Expects Next

Good evening. A word about the state of things.

The database wars are over. PostgreSQL won.

This is not a bold claim. It is simply what the numbers say. Stack Overflow's 2024 Developer Survey placed PostgreSQL as the most-used database for the third consecutive year, at 52% adoption among professional developers. DB-Engines has tracked its steady ascent since 2018. Every major cloud provider now offers a managed PostgreSQL service — several offer more than one. Supabase, Neon, CockroachDB, YugabyteDB, AlloyDB, and Timescale are all built on it or speak its wire protocol.

The question "which database should I use?" has, for most applications, a default answer. And it is a good answer. PostgreSQL is remarkable software — standards-compliant, extensible, reliable, and genuinely free.

But winning creates its own problems. And PostgreSQL's victory has exposed a gap that, I must confess, has been bothering me for some time.

The numbers, since numbers are what settle matters

Allow me to be specific, because specificity is what distinguishes observation from opinion.

From fourth place at 36% to first place at 52% in five years. That is not a trend. That is a verdict.

DB-Engines named PostgreSQL its "DBMS of the Year" four times — more than any other database in the award's history. The Timescale State of PostgreSQL 2024 survey found that 72% of respondents used PostgreSQL as their primary database. Not "one of several." Primary.

The trajectory is particularly telling because it happened against the prevailing narrative. From 2015 to 2019, the industry was convinced that the future belonged to purpose-built databases: MongoDB for documents, Redis for caching, Elasticsearch for search, ClickHouse for analytics, DynamoDB for scale. Each workload would have its own specialist. The polyglot persistence future was meant to be inevitable.

PostgreSQL's response to this thesis was not to argue with it. It was to quietly absorb it. JSONB made the document database argument less compelling. Full-text search with tsvector and tsquery reduced the case for a separate search engine. pg_trgm added fuzzy matching. PostGIS handled geospatial. TimescaleDB extended it for time-series. pgvector brought vector similarity search into the fold. Materialized views, LISTEN/NOTIFY, and SKIP LOCKED absorbed caching and queuing patterns. TABLESAMPLE, window functions, CTEs, lateral joins — each release added capabilities that made the "you need a separate tool for that" argument slightly less persuasive.

The polyglot persistence movement is not dead. But its scope has narrowed considerably. For most applications, the answer to "do I need a separate database for this?" is now "probably not, if you're using PostgreSQL well." And "well" is the operative word. We shall return to it.

How did PostgreSQL win? It is worth understanding.

It won on merit, which is the only way that matters.

PostgreSQL offered what MySQL could not: proper SQL compliance, sophisticated query planning, transactional DDL, extensibility through custom types and functions, and a development community that prioritized correctness over speed-to-market. When developers needed JSON support, PostgreSQL added JSONB — a binary format with GIN indexing that made it genuinely useful, not merely present. When they needed full-text search, PostgreSQL built it into the core. When they needed upserts, PostgreSQL gave them INSERT ... ON CONFLICT with proper concurrency semantics.

But merit alone does not win markets. Plenty of superior technologies have lost to inferior ones with better distribution. PostgreSQL needed something else, and it got two things simultaneously.

The first was the cloud. AWS RDS, Google Cloud SQL, and Azure Database for PostgreSQL removed the operational barrier that once kept teams on simpler databases. You no longer needed a DBA to run PostgreSQL in production. You needed a credit card. The operational complexity that had been PostgreSQL's one genuine weakness — compared to MySQL's "install and forget" simplicity — vanished overnight. Managed PostgreSQL was as easy to provision as managed MySQL, and considerably more capable once provisioned.

The second was the ecosystem. ORMs defaulted to PostgreSQL. Tutorials used it. Boot camps taught it. Rails, Django, and Laravel all supported PostgreSQL from the start, but by 2020, their communities had shifted from "also supports PostgreSQL" to "assumes PostgreSQL." Prisma, Drizzle, and the newer ORMs were PostgreSQL-first from day one. The decision became self-reinforcing: PostgreSQL had the most users, which produced the best tooling, which attracted more users.

And then the PostgreSQL-compatible databases appeared. Aurora PostgreSQL proved you could build a distributed database on the PostgreSQL wire protocol. AlloyDB proved Google thought so too. Neon proved you could build serverless Postgres. CockroachDB, YugabyteDB, and others adopted the wire protocol as a distribution strategy. The protocol itself became the standard — not just the database.

This is the clearest signal that PostgreSQL has won. When competitors adopt your interface rather than competing with it, the war is over. You are no longer a database. You are the platform.

A candid word about what PostgreSQL is not

I should be forthcoming about this, because pretending PostgreSQL has no weaknesses would be an embarrassment to me and a disservice to you.

PostgreSQL is not the fastest at everything it does. It is the most capable at the widest range of things, which is a different and, I would argue, more useful property. But the distinction matters.

For sub-millisecond key-value lookups at extreme concurrency — tens of thousands of reads per second on hot keys — Redis is genuinely faster. Its data lives in memory, its protocol has less overhead, and its single-threaded event loop avoids the coordination costs that PostgreSQL pays for MVCC. If you are building a session store, a rate limiter, or a leaderboard that needs microsecond response times at 50,000 requests per second, keep Redis. I have no quarrel with Redis deployed for the problems Redis was designed to solve.

For full-text search at scale — millions of documents with complex relevance scoring, faceted search, typo tolerance, and real-time indexing — Elasticsearch and Meilisearch are purpose-built and it shows. PostgreSQL's tsvector is remarkably good for a feature that is essentially a side project of a relational database, but "remarkably good for a side project" is not the same as "best in class." If search is your product's primary feature, evaluate the specialists.

For analytical queries over billions of rows — columnar scans, OLAP aggregations, data warehouse workloads — column-oriented databases like ClickHouse, DuckDB, and BigQuery process these queries orders of magnitude faster. PostgreSQL is row-oriented by design, and no amount of indexing changes the fundamental scan pattern for wide analytical queries across massive datasets.

For globally distributed writes with strong consistency — Spanner, CockroachDB, and YugabyteDB offer multi-region write capabilities that PostgreSQL's single-primary architecture cannot match without significant application-level complexity.

These are real boundaries. Acknowledging them is what makes the rest of this argument credible. For the overwhelming majority of applications — the ones that need a reliable, capable, general-purpose database to store and retrieve structured data — PostgreSQL is the correct choice. For the specialized edges, the specialists still earn their keep.

A waiter who overstates his case is no waiter at all.

What gap did winning expose?

PostgreSQL is an extraordinary executor. You give it a query, it executes it. Faithfully, correctly, every single time. It is, in this respect, the perfect servant — it does exactly what you ask, never more.

And therein lies the problem.

-- Your application runs this 50,000 times per day:
SELECT u.name, COUNT(o.id), SUM(o.total)
FROM users u
JOIN orders o ON o.user_id = u.id
WHERE o.created_at > NOW() - INTERVAL '24 hours'
GROUP BY u.name;

-- PostgreSQL will execute it faithfully, every time.
-- It will not suggest an index. It will not cache the result.
-- It will not mention that you've asked the same question
-- 50,000 times and the answer hasn't changed in 5 minutes.

PostgreSQL will not optimize your workload. It will optimize individual queries within the constraints of the data it has — table statistics, available indexes, join strategies — and it does this remarkably well. The query planner is a genuine marvel of engineering, considering thousands of possible execution paths and choosing the cheapest one in milliseconds. I do not fault the planner. I admire the planner.

But the planner operates within a single query, at a single moment. It cannot step back and observe patterns. It will not notice that you are running the same expensive aggregation 50,000 times per day with the same result. It will not create a materialized view on your behalf. It will not add the missing index that would turn a 400ms sequential scan into a 2ms index scan. It will not mention that 60% of your database's total CPU time is consumed by three queries that could each be fixed in minutes.

This is not a criticism of PostgreSQL. It is, by design, a general-purpose database engine, not a workload optimizer. Asking PostgreSQL to optimize your workload is like asking a master chef to also do the shopping, plan the menu, and manage the household budget. The chef's job is to cook whatever you bring to the kitchen, and to cook it brilliantly. The question of what to cook, and how often, and whether you really need to prepare the same elaborate dish from scratch fifty thousand times when the answer hasn't changed in five minutes — that is a different job entirely.

That knowledge — what your application actually needs from the database, and whether it is asking for it efficiently — exists in the space between the application and the database. A space that, until recently, was occupied by exactly no one.

The tooling gap, examined properly

Consider the tooling that exists today for PostgreSQL in production. It is extensive. It is, in places, excellent. And it is categorically incomplete.

• Monitoring — pganalyze, Datadog, pg_stat_monitor, pgwatch2. These tell you what is slow. They do not fix it. They are the smoke detector in a house that has no fire department.
• Connection pooling — PgBouncer, pgcat, Supavisor. These manage connections with admirable efficiency. They do not touch queries. A perfectly pooled connection that delivers a badly written query is still a badly written query, arriving more efficiently.
• Query analysis — pg_stat_statements, auto_explain, pgBadger. These surface data. They do not act on it. They are, if you will permit the metaphor, the waiter who catalogues every creak in the floorboards but never calls the carpenter.
• Index advisors — Dexter, pg_qualstats, HypoPG. These suggest indexes. They do not create them. They do not manage them. They do not remove the ones that are no longer useful. The advice is sound; the execution is left as an exercise for the reader.
• ORMs — Django, Rails, SQLAlchemy, Prisma, Drizzle. These generate queries. They do not evaluate whether those queries are efficient. An ORM that generates an N+1 pattern across 200 rows will do so consistently and without remorse, 100,000 times per day, until a human notices.

Each of these tools is excellent at its specific job. None of them close the loop. The cycle is instructive:

-- The human optimization loop, as it exists today:

-- Step 1: Monitoring tool fires an alert
--   "Query xyz exceeded P95 latency threshold"

-- Step 2: Engineer investigates (hours to days later)
--   EXPLAIN (ANALYZE, BUFFERS) SELECT ...

-- Step 3: Engineer identifies the fix
--   CREATE INDEX idx_orders_created_at ON orders (created_at);

-- Step 4: Engineer deploys the fix
--   Migration, PR review, staging test, production deploy

-- Step 5: Another query degrades
--   Return to Step 1

-- Average cycle time: 3-14 days
-- During which the slow query runs ~150,000 more times

This cycle has a name in other industries. It is called "reactive maintenance." The factory that waits for machines to break before fixing them. The household that waits for the pipes to burst before calling the plumber. It is universally regarded as the most expensive way to maintain anything, and it is the default — the only — mode of PostgreSQL query optimization in most production environments.

The monitoring tool fires an alert. A human investigates. The human identifies the fix. The human implements the fix through the deployment pipeline. The fix reaches production days after the problem began. And then another query degrades. Another table grows. Another ORM generates another N+1 pattern. The human cycle begins again.

This is the gap. Not a gap in PostgreSQL itself — PostgreSQL is extraordinary. Not a gap in any individual tool — the tools are good at what they do. The gap is that nobody closes the loop. The space between "we know what's wrong" and "it's fixed" is occupied by a human, a deployment pipeline, and a prayer that someone notices before the next on-call rotation.

Why this gap exists: the history matters

The gap is not an accident. It is the natural consequence of how database tooling evolved.

In the early days of relational databases — the 1990s and early 2000s — every serious deployment had a DBA. The DBA was the loop-closer. They monitored queries, identified problems, created indexes, designed materialized views, tuned autovacuum, managed connection pools. The gap between observation and action was a person, and that person was on staff.

Then two things happened simultaneously. Cloud databases eliminated the operational DBA role for small and medium teams. And application development exploded in scale — more teams, more applications, more databases, all without the DBA who used to maintain them.

The monitoring tools appeared to fill the gap left by the DBA. But they filled only the observation half. The DBA did two things: observe and act. The tools replaced the observation. Nobody replaced the action.

For large companies, this is manageable. They can afford a database team, or a platform engineering team, or at minimum a senior engineer who understands EXPLAIN ANALYZE output and can diagnose a missing index. The human loop-closer exists. They are expensive and in short supply, but they exist.

For the vast majority of teams — the startup with five engineers, the consultancy with three, the solo founder with one — the loop-closer does not exist. There is no one on staff who knows what a B-tree index looks like in an EXPLAIN plan, or when a materialized view is appropriate, or why autovacuum is falling behind. These teams use PostgreSQL because it is the right choice. They use it badly because the tooling assumes expertise that they do not have and cannot afford to hire.

The advice to hire a DBA arrives most enthusiastically from those who have never tried to hire one. The global shortage of experienced PostgreSQL specialists is well documented and shows no signs of resolving. You cannot close the gap with humans that do not exist in sufficient quantity.

What the capabilities already contain

Here is what I find most frustrating about this gap: PostgreSQL already has the building blocks. Every one of them. They are sitting in the database, fully operational, waiting to be used.

-- PostgreSQL has had this capability since version 9.3 (2013):
CREATE MATERIALIZED VIEW mv_daily_user_sales AS
SELECT u.name, COUNT(o.id) AS order_count, SUM(o.total) AS revenue
FROM users u
JOIN orders o ON o.user_id = u.id
WHERE o.created_at > NOW() - INTERVAL '24 hours'
GROUP BY u.name;

-- Refresh it when the data changes:
REFRESH MATERIALIZED VIEW CONCURRENTLY mv_daily_user_sales;

-- Now your 50,000 daily queries read pre-computed results:
-- 412ms → 0.3ms. Same hardware. Same data. Same answer.

Materialized views. Indexes — B-tree, GIN, GiST, BRIN, hash. Partial indexes for targeted subsets. Covering indexes with INCLUDE columns for index-only scans. REFRESH MATERIALIZED VIEW CONCURRENTLY for zero-downtime updates. The wire protocol for transparent proxying. pg_stat_statements for workload analysis. pg_stat_user_tables for access pattern detection.

PostgreSQL is not missing features. It is missing orchestration. The capabilities exist. What does not exist — or did not, until recently — is something that watches the traffic, identifies which capabilities would help, deploys them, and maintains them as the workload changes.

This is analogous to a household that possesses every cleaning supply, every tool, every piece of equipment — neatly organized in the cellar — but employs no one to use them. The supplies do not clean the house by themselves. They need someone to assess what needs doing, select the right tool, apply it, and come back next week to do it again.

What the next layer looks like

The logical next step is a layer that sits between the application and the database — one that observes actual query patterns, identifies optimization opportunities, and applies them without requiring code changes or ongoing human intervention.

This is not theoretical. The building blocks, as I have just described, already exist. What has been missing is the orchestration — something that watches, decides, and acts. The observe-decide-act loop that the DBA used to provide, automated and continuous.

The requirements for this layer are specific, and I should be precise about them because precision is what separates a useful idea from a marketing slide:

• Transparent. It must work with any application, any ORM, any language — by operating at the PostgreSQL wire protocol level, not the application level. If it requires code changes, it is not solving the problem for the teams that need it most. The application should not know it exists. The queries go in, the results come back, and the only observable difference is that they come back faster.
• Automatic. It must not require a human to identify which queries need optimization. The patterns are in the traffic; they should be discovered, not configured. A query that runs 50,000 times per day with a 400ms execution time and a stable result set is obviously a candidate for materialization. The evidence is in the data. A system that requires a human to notice this is a system that has missed the point.
• Self-maintaining. When query patterns change — as they always do — the optimizations must adapt. When a new feature ships and new queries appear, the system should detect and optimize them. When old queries disappear, the materialized views and indexes that served them should be cleaned up. Unused views are not just waste; they are active cost — each one consumes storage and refresh cycles. A well-run household does not employ three waiters where one suffices.
• Safe. It must never return incorrect data. This is the non-negotiable requirement, and the one that makes the problem genuinely hard. The optimization layer is invisible to the application; if it makes a mistake, no one catches it. Returning a stale materialized view result when the underlying data has changed is not a performance optimization — it is a bug. The system must know when to serve cached results and when to fall through to the source of truth, and it must never get this wrong.
• Observable. Invisible to the application does not mean invisible to the operator. The system should explain what it has done, why, and what the measured impact was. A waiter who rearranges the household without informing anyone is not a waiter. He is a poltergeist.

This is, as it happens, the problem Gold Lapel was built to solve. A transparent Rust proxy that watches your PostgreSQL traffic, identifies patterns worth optimizing, creates the appropriate indexes and materialized views, and rewrites queries to use them — continuously and automatically. Not a monitoring tool. Not an advisory tool. The thing that closes the loop.

I mention this not as a sales pitch — the Waiter does not hard-sell — but as an acknowledgment that the gap I have described is the one we set out to address. You are welcome to evaluate whether we have addressed it well. The architecture is documented in detail, and the honest assessment of what it does and does not do is available for your inspection.

The honest counterpoint: why this layer did not exist sooner

If this idea is so obviously correct, one might reasonably ask why it took until now to build it. The question deserves a direct answer.

The primary reason is that automatic query optimization is genuinely difficult to do safely. The moment you begin rewriting queries and serving results from materialized views instead of base tables, you take responsibility for data correctness. A monitoring tool that misidentifies a slow query has no consequence — nobody acts on it until a human verifies. An optimization proxy that serves stale data has immediate, invisible, and potentially severe consequences.

The safety bar is extraordinarily high. You need write-aware invalidation — not TTL-based timers that hope the data has not changed, but actual observation of writes flowing through the proxy, triggering immediate invalidation of affected cached results. You need to understand which materialized views are affected by which writes, in real time, at wire-protocol speed. This is not a weekend project.

The second reason is economic. When every team had a DBA, the automation was the DBA. The gap only became acute when cloud databases removed the DBA from the equation and the number of unoptimized PostgreSQL deployments exploded. The problem became large enough to justify a product only recently.

The third reason is technical. Building a transparent proxy that speaks the full PostgreSQL wire protocol — including prepared statements, extended query protocol, COPY, LISTEN/NOTIFY — is a significant systems programming effort. It requires a language with the performance characteristics and memory safety guarantees to sit in the hot path of every database query without adding meaningful latency. Rust, as it happens, is rather good at this. Its availability and maturity in the last few years made certain architectural choices practical that would have been inadvisable earlier.

I offer these reasons not as excuses but as context. The gap existed for years because closing it safely is hard. That does not make the gap less real. It makes the solution more valuable.

What happens next

PostgreSQL's dominance is not going to reverse. If anything, the PostgreSQL-compatible databases — Aurora, AlloyDB, Neon — are reinforcing it by proving that the wire protocol is the standard, not just the database itself. When Google builds a new database product and chooses PostgreSQL compatibility as its interface, that is not a market trend. That is a market fact.

The era of choosing a database is ending, for most applications, because the choice has been made. PostgreSQL. The era of choosing how well you use that database is just beginning.

The question is no longer "which database?" It is "how well are you using it?" And for most teams, the honest answer is: not nearly as well as they could be. Not because they lack skill or dedication, but because the tools between the application and the database have not kept pace with either. The database has gotten faster every year. The applications have gotten more complex every year. The space between them has remained essentially vacant.

That gap is where the next wave of database tooling will live. Not replacing PostgreSQL — that would be both foolish and unnecessary. Building on top of it. Making the extraordinary database genuinely effortless to use well. Closing the loop between observation and action that has been open since the last DBA left the building.

PostgreSQL won the database war. What it needs now is not a better engine — the engine is magnificent. What it needs is the optimization layer that treats every query as an opportunity to learn, every pattern as a signal, and every wasted sequential scan as a matter requiring immediate attention.

The household has high expectations. Quite rightly so. And I believe, if you will allow me the observation, that those expectations are about to be met.