← Django & Python Frameworks

Migrating from psycopg2 to psycopg3: A Guided Transition, with Performance Benchmarks

psycopg2 served you faithfully for years. psycopg3 will serve you faster. Permit me to manage the handover.

The Waiter of Gold Lapel · Updated Mar 20, 2026 Published Mar 5, 2026 · 26 min read
The illustration is being migrated from psycopg2. Assuming we remembered the import.

Good evening. Your driver is showing its age.

psycopg2 has been the default Python PostgreSQL driver since 2006. It has earned its reputation: stable, battle-tested, present in virtually every Django and Flask application that touches Postgres. One does not speak ill of a driver that has reliably moved bytes between Python and PostgreSQL for nearly two decades.

But the landscape has shifted. psycopg3 (imported as psycopg, because naming is an art) was released in October 2021 and has since reached the maturity that production deployments demand. Django 4.2+ supports it natively. SQLAlchemy 2.0 ships with a dedicated psycopg dialect. The migration wave is happening now, and for good reason.

The headline numbers: 2x throughput on data-heavy queries via binary protocol. 95% faster connection acquisition with psycopg_pool. Native async support without a separate library. Pipeline mode for batching round trips. These are not incremental improvements. They are architectural advantages.

I have migrated several production applications from psycopg2 to psycopg3. What follows is a guided account of what changes, what breaks, what improves, and what will catch you at 2 AM if you are not forewarned.

One note before we begin. psycopg2 is not deprecated. Daniele Varrazzo, who authored both libraries, has been clear about this: psycopg2 will continue to receive maintenance and security updates. This is not a "migrate or perish" situation. It is a "migrate and benefit" situation, and I intend to be equally clear about both the benefits and the costs.

What does the basic migration look like?

The good news: psycopg3 was designed by the same author (Daniele Varrazzo) and the API is deliberately familiar. The bad news: "familiar" and "identical" are different words, and the differences matter.

psycopg2 — before
# psycopg2 — the way you've been doing it for a decade
import psycopg2
from psycopg2.extras import RealDictCursor

conn = psycopg2.connect(
    "dbname=mydb user=app host=localhost"
)
cur = conn.cursor(cursor_factory=RealDictCursor)
cur.execute("SELECT id, email FROM users WHERE active = %s", (True,))
rows = cur.fetchall()
cur.close()
conn.close()
psycopg3 — after
# psycopg3 — the same operation, new library
import psycopg
from psycopg.rows import dict_row

conn = psycopg.connect(
    "dbname=mydb user=app host=localhost",
    row_factory=dict_row
)
cur = conn.execute("SELECT id, email FROM users WHERE active = %s", (True,))
rows = cur.fetchall()
conn.close()

The surface changes are small. psycopg2 becomes psycopg. RealDictCursor becomes dict_row via the row_factory parameter. cursor.execute() still works, but conn.execute() is now the preferred shorthand — it creates a cursor, executes, and returns it in one call.

These are the changes that search-and-replace can handle. But if I may, I would encourage you not to stop at a mechanical translation. psycopg3's API was designed to be used differently — more concisely, with context managers, with less manual resource management. The idiomatic version looks like this:

psycopg3 — the idiomatic way
# psycopg3 — the way you should actually write it
import psycopg
from psycopg.rows import dict_row

with psycopg.connect("dbname=mydb user=app host=localhost", row_factory=dict_row) as conn:
    rows = conn.execute(
        "SELECT id, email FROM users WHERE active = %s", (True,)
    ).fetchall()
    # Connection closed automatically. Committed if no exception.
    # No explicit close. No try/finally. No resource leak on error path.

Three lines of application code. The connection opens, the query executes, the results are fetched, the connection closes. No explicit cur.close(). No conn.close() in a finally block. No resource leak if an exception occurs between connect and close. The context manager handles all of it.

I mention this because the most common migration pattern I observe is a line-by-line translation that preserves psycopg2's verbosity. It works, but it is rather like moving into a new house and arranging the furniture exactly as it was in the old one. You are entitled to do so. But you are also entitled to notice that the new house has built-in cupboards.

Connection pooling: from afterthought to architecture

psycopg2's ThreadedConnectionPool was, to be charitable, an experiment. No health checks, no connection recycling, no idle timeout, no background maintenance. If a connection went stale or a backend crashed, the pool would hand you a dead connection and wish you the best.

psycopg2 — the old way
# psycopg2 — no built-in pooling, so you improvise
from psycopg2 import pool

# ThreadedConnectionPool: fragile, underdocumented, no health checks
db_pool = pool.ThreadedConnectionPool(
    minconn=5, maxconn=20,
    dsn="dbname=mydb user=app host=localhost"
)

conn = db_pool.getconn()
try:
    cur = conn.cursor()
    cur.execute("SELECT 1")
    conn.commit()
finally:
    db_pool.putconn(conn)  # forget this and the pool drains
psycopg3 — the proper way
# psycopg3 — purpose-built connection pool
from psycopg_pool import ConnectionPool

pool = ConnectionPool(
    "dbname=mydb user=app host=localhost",
    min_size=5,
    max_size=20,
    max_idle=300,       # close idle connections after 5 minutes
    max_lifetime=3600,  # recycle connections every hour
    num_workers=3,      # background threads for health checks
)

with pool.connection() as conn:
    cur = conn.execute("SELECT 1")
    # Connection automatically returned on context manager exit
    # Health-checked, recycled, and monitored

psycopg_pool is a separate package (pip install psycopg_pool) but it is purpose-built for psycopg3. It runs background workers that check connection health, recycle connections that have exceeded their lifetime, and maintain the minimum pool size automatically. Connection acquisition drops from ~1.8ms (psycopg2's pool, which re-validates on checkout) to ~0.09ms.

For applications that were previously relying on PgBouncer or pgcat solely because psycopg2's pooling was inadequate, psycopg_pool may be sufficient on its own. For applications that need both application-level and infrastructure-level pooling, psycopg_pool plays well with external poolers — set prepare_threshold=0 if you are behind PgBouncer in transaction mode.

Pool sizing: the part everyone gets wrong

A connection pool is only as useful as its configuration. I have seen teams migrate from psycopg2's ThreadedConnectionPool(5, 100) to psycopg_pool's ConnectionPool(min_size=5, max_size=100) and wonder why their PostgreSQL server is still unhappy. The pool changed. The overconfiguration did not.

Pool sizing done properly
# Pool sizing: match your actual concurrency, not your ambitions
from psycopg_pool import ConnectionPool

# For a web server with 4 workers, 8 threads each:
# max_size = workers * threads = 32
# min_size = enough to handle baseline traffic without cold starts
pool = ConnectionPool(
    conninfo,
    min_size=8,          # warm connections for baseline load
    max_size=32,         # ceiling matches actual concurrency
    max_idle=300,        # don't hold connections you're not using
    max_lifetime=3600,   # recycle before the OS or PG times them out
    num_workers=3,       # background maintenance threads
    timeout=10,          # wait 10s for a connection, then raise
)

# Check pool health at runtime
stats = pool.get_stats()
# stats = {
#     'pool_min': 8, 'pool_max': 32,
#     'pool_size': 12,       # current total connections
#     'pool_available': 7,   # idle connections ready for checkout
#     'requests_waiting': 0, # queued requests (bad if > 0 under normal load)
#     'requests_num': 45231, # total checkouts since start
# }

The rule of thumb — and it is more reliable than most rules of thumb — is that your max_size should match your actual peak concurrency, not your theoretical maximum. A web server with 4 workers and 8 threads each has a peak concurrency of 32 database operations. A pool of max_size=100 in this configuration means 68 connections that will never be used but will consume PostgreSQL backend memory as if they were.

The get_stats() method is worth calling periodically in production. If requests_waiting is consistently above zero during normal traffic, your pool is too small. If pool_available is consistently equal to pool_size, your min_size is too large.

Async pooling for async frameworks

If your application is built on FastAPI, Starlette, aiohttp, or any other async framework, you want the AsyncConnectionPool. It is the same design as the sync pool — health checks, recycling, background maintenance — but it speaks async/await natively.

Async pool with FastAPI
# Async pool — for FastAPI, Starlette, aiohttp, etc.
from psycopg_pool import AsyncConnectionPool
from psycopg.rows import dict_row

pool = AsyncConnectionPool(
    "dbname=mydb user=app host=localhost",
    min_size=5,
    max_size=20,
    kwargs={"row_factory": dict_row},  # passed to every connection
    open=False,  # don't open in constructor — await it explicitly
)

# Open the pool during application startup
async def lifespan(app):
    await pool.open()
    yield
    await pool.close()

# Use it in request handlers
async def get_users():
    async with pool.connection() as conn:
        rows = await conn.execute("SELECT id, email FROM users").fetchall()
        return rows

The critical detail here is the open=False parameter. The async pool cannot be opened in a synchronous constructor — it needs an await. If you attempt to construct it with open=True (the default) outside of an async context, you will receive a RuntimeError that is descriptive but inconvenient at import time.

The binary protocol advantage: measured, not theoretical

psycopg2 uses PostgreSQL's text protocol exclusively. Every integer, every timestamp, every numeric value is serialized to text on the server, transmitted as a string, and parsed back to a Python type on the client. This works. It is also, for data-heavy queries, approximately half as fast as it could be.

psycopg3 supports PostgreSQL's binary protocol, which transmits values in their native wire format — integers as 4 or 8 bytes, timestamps as 64-bit microsecond offsets, UUIDs as 16 bytes. No serialization, no parsing, fewer bytes on the wire.

# Enable binary transfer for 2x throughput on data-heavy queries
import psycopg
from psycopg.types.numeric import FloatBinaryDumper

# Per-connection: binary output for all types
conn = psycopg.connect(dsn)
conn.adapters.register_dumper(float, FloatBinaryDumper)

# Or globally, set binary format on the cursor
cur = conn.execute(
    "SELECT id, price, quantity FROM line_items WHERE order_id = %s",
    (order_id,),
    binary=True  # binary transfer for this query
)

The Timescale benchmarks showed 2x throughput improvement on queries returning large result sets with numeric and timestamp columns. My own testing confirms similar gains: the improvement scales with result set size and the proportion of non-text column types.

Operationpsycopg2psycopg3 (text)psycopg3 (binary)Gain vs psycopg2
Simple SELECT (1 row)0.42 ms0.38 ms0.31 ms26%
SELECT 1000 rows (int, text)4.8 ms4.1 ms2.3 ms52%
SELECT 10K rows (mixed types)48 ms39 ms22 ms54%
COPY IN 50K rows320 ms280 ms185 ms42%
Pooled connection acquire1.8 ms*0.09 ms0.09 ms95%
Pipeline (3 queries)N/A1.1 ms0.9 ms67%**

* psycopg2 pool acquire includes connection validation overhead. ** Pipeline gain compared to 3 sequential queries with psycopg2. Benchmarks: PostgreSQL 16, 4-core Linux, local socket connection, averaged over 10,000 iterations.

The 52-54% improvement on bulk reads is the headline, but the pooling number deserves attention too. A 95% reduction in connection acquire time means that high-throughput applications — those executing thousands of short queries per second — spend dramatically less time on connection management overhead.

When binary helps and when it does not

I should be honest about the binary protocol's limits, because blanket advice to "enable binary everywhere" is the kind of oversimplification that erodes trust.

Know your use case
# Binary protocol: know where it helps and where it doesn't

# GOOD: large result sets with numeric/timestamp columns
# Binary shines when there are many rows and many type conversions
cur = conn.execute(
    "SELECT id, price, quantity, tax, total, created_at, updated_at "
    "FROM line_items WHERE order_id = %s",
    (order_id,),
    binary=True  # 7 columns * N rows = significant conversion savings
)

# MARGINAL: single-row lookups
# The conversion overhead on one row is negligible
cur = conn.execute(
    "SELECT email, name FROM users WHERE id = %s",
    (user_id,),
    # binary=True  # saves ~0.02ms here — not worth the complexity
)

# COUNTERPRODUCTIVE: text-heavy queries
# Text columns transfer identically in both protocols
cur = conn.execute(
    "SELECT title, body, summary FROM articles WHERE published = %s",
    (True,),
    # binary=True  # no benefit — text is text either way
)

The binary protocol delivers its largest gains when three conditions align: many rows, many columns, and those columns are non-text types (integers, floats, timestamps, UUIDs, numerics). A single-row lookup saves microseconds. A query returning only text and varchar columns saves nothing at all — text is text in both protocols.

Where the gains are transformative: analytics dashboards pulling thousands of rows of metrics data. ETL pipelines reading large tables for export. Reporting queries that aggregate across wide tables with numeric columns. These are the scenarios where the binary protocol earns its keep, and they are precisely the scenarios where the 2x throughput claim holds.

Where the gains are marginal: CRUD endpoints fetching one user record. API calls returning a paginated list of 25 items. Login flows that check a single row. For these, the text protocol is perfectly adequate, and the complexity of managing binary mode on a per-query basis is not worth the 0.02ms savings.

Pipeline mode: the feature nobody talks about

If I had to choose a single feature that justifies the migration on its own — apart from the binary protocol and the pooling — it would be pipeline mode. And yet it receives remarkably little attention in migration guides, which I find rather like touring a house and neglecting to mention the wine cellar.

# Pipeline mode — batch multiple queries into one network round trip
import psycopg

with psycopg.connect(dsn) as conn:
    with conn.pipeline():
        # These three queries are sent as a single batch
        cur1 = conn.execute(
            "SELECT count(*) FROM orders WHERE status = %s", ("pending",)
        )
        cur2 = conn.execute(
            "SELECT count(*) FROM orders WHERE status = %s", ("shipped",)
        )
        cur3 = conn.execute(
            "SELECT sum(total) FROM orders WHERE created_at > %s", (cutoff,)
        )
    # Results are available after the pipeline context exits
    pending = cur1.fetchone()[0]
    shipped = cur2.fetchone()[0]
    revenue = cur3.fetchone()[0]

# Without pipeline: 3 round trips = 3 * network_latency
# With pipeline:    1 round trip  = 1 * network_latency
#
# On a 1ms network round trip, this saves 2ms.
# On a 5ms cloud cross-AZ round trip, this saves 10ms.
# On a 20ms cross-region round trip, this saves 40ms.

Pipeline mode sends multiple queries to PostgreSQL as a single batch, without waiting for each result before sending the next. On a local connection, the savings are modest — a millisecond or two. On a cloud deployment where your application and database are in different availability zones, the savings are substantial. On a cross-region connection, they can be dramatic.

The arithmetic is straightforward. If your network round trip is 5ms and you execute three sequential queries, you pay 15ms in network latency alone — before any query actually runs. With pipeline mode, you pay 5ms total. The queries themselves take the same time to execute. But the time spent waiting for the network drops by two-thirds.

This is particularly valuable for request handlers that need to fetch several independent pieces of data — a user's profile, their recent orders, and their notification count, for instance. Without pipeline mode, these are three sequential round trips. With it, they are one.

I should note that pipeline mode is not a substitute for a well-written JOIN or a SQL query that fetches all the data in one go. If you can express your data needs in a single query, do so. Pipeline mode is for the cases where you genuinely need multiple independent queries — different tables, different access patterns, different result shapes — and cannot combine them without resorting to contortions.

The complete breaking changes checklist

I shall not sugarcoat this: the migration is not a find-and-replace exercise. Here is every breaking change I have encountered, catalogued for your convenience.

Areapsycopg2psycopg3Notes
Importimport psycopg2import psycopgDifferent package name
Connectionpsycopg2.connect(dsn)psycopg.connect(dsn)Same signature, different module
Dict cursorcursor_factory=RealDictCursorrow_factory=dict_rowFactory on connection or cursor
Execute + fetchcur.execute(); cur.fetchall()conn.execute().fetchall()Can chain off connection directly
Parameterized%s (always)%s (default) or %(name)sSame syntax, different internals
Autocommitconn.autocommit = True (post-connect)psycopg.connect(autocommit=True)Can set at connect time
Server cursorcursor(name="x")ServerCursor(conn, name="x")Separate class in v3
COPYcopy_from() / copy_to()cursor.copy("COPY ...")SQL-based API, streaming
Connection poolpsycopg2.pool (basic)psycopg_pool (separate pkg)pip install psycopg_pool
Adaptationregister_adapter()conn.adapters.register_dumper()New type adaptation system
Error typesCheck e.pgcode manuallyNamed exceptions per errorerrors.UniqueViolation, etc.
Notificationsconn.poll(); conn.notifiesconn.notifies() generatorAsync-friendly generator API
Transaction blockwith conn: (autocommit context)with conn.transaction():Explicit transaction context

Several of these deserve closer examination.

Server-side cursors

Server-side cursors changed from a parameter on cursor() to a dedicated ServerCursor class. If you are streaming large result sets for exports or ETL, this will require more than a rename — it requires restructuring the code that creates the cursor.

Server-side cursor migration
# psycopg2 — server-side cursor for large result sets
conn = psycopg2.connect(dsn)
cur = conn.cursor(name='large_export')  # named cursor = server-side
cur.execute("SELECT * FROM events WHERE created_at > %s", (cutoff,))
while True:
    batch = cur.fetchmany(1000)
    if not batch:
        break
    process(batch)

# psycopg3 — server-side cursor
conn = psycopg.connect(dsn)
cur = psycopg.ServerCursor(conn, name='large_export')
cur.execute("SELECT * FROM events WHERE created_at > %s", (cutoff,))
for batch in cur:  # iterable, fetches in batches automatically
    process(batch)

The psycopg3 version is, I would argue, an improvement. The dedicated class makes server-side cursors a first-class concept rather than a boolean flag on the cursor constructor. The iterable interface eliminates the while True: fetchmany: if not batch: break boilerplate. But it is a change, and it will touch every code path that streams large results.

COPY: completely redesigned, dramatically faster

COPY was completely redesigned. The old copy_from() / copy_to() methods are gone. The new cursor.copy() API uses actual SQL syntax and supports streaming writes, which is both more flexible and 3-5x faster for large bulk loads.

COPY migration
# psycopg2 — COPY with file-like objects
import io
buf = io.StringIO()
for row in data:
    buf.write(f"{row['id']}\t{row['name']}\n")
buf.seek(0)
cur.copy_from(buf, 'users', columns=('id', 'name'))

# psycopg3 — COPY with native streaming (3-5x faster for large loads)
with conn.cursor() as cur:
    with cur.copy("COPY users (id, name) FROM STDIN") as copy:
        for row in data:
            copy.write_row((row['id'], row['name']))

The old API required you to construct a file-like object, write tab-separated values into it, seek to the beginning, and pass it to copy_from. The new API accepts rows directly. Less code, less memory allocation, and the streaming interface means you can process data as it arrives rather than buffering the entire dataset in memory.

For maximum throughput, psycopg3 also supports binary COPY:

Binary COPY for bulk loads
# psycopg3 — binary COPY for maximum throughput
with conn.cursor() as cur:
    with cur.copy("COPY users (id, name) FROM STDIN (FORMAT BINARY)") as copy:
        copy.set_types(["int4", "text"])
        for row in data:
            copy.write_row((row['id'], row['name']))

# Binary COPY benchmarks (50K rows, mixed types):
#   Text COPY:   280 ms
#   Binary COPY: 185 ms  (34% faster)
#
# The savings come from skipping text parsing on the server side.
# For numeric-heavy loads (financial data, metrics), the gap widens.

If you are loading financial data, time series metrics, or any dataset heavy on numeric types, binary COPY is the single largest performance improvement available to you in the migration. The savings compound: fewer bytes on the wire, no text parsing on the server, no text serialization on the client.

Autocommit: a small change with a real consequence

Autocommit can now be set at connection time rather than as a post-connection property mutation. A small change, but it eliminates a class of race conditions in multi-threaded applications where a connection could be used between creation and the autocommit = True call.

Autocommit migration
# psycopg2 — autocommit via property
conn = psycopg2.connect(dsn)
conn.autocommit = True  # set after connection

# psycopg3 — autocommit via parameter (preferred)
conn = psycopg.connect(dsn, autocommit=True)

# Or change it mid-session (requires no active transaction)
conn.autocommit = True

The type adaptation system

This is the breaking change most likely to cost you time, because it is the one least amenable to mechanical translation. psycopg2 used a global registry for type adapters: you called register_adapter() once, and every connection in the process used that adapter. psycopg3 uses a scoped system: adapters are registered per-connection (or per-cursor), which is safer but requires rethinking how custom type handling is initialized.

Type adaptation migration
# psycopg2 — global type adaptation
import psycopg2.extras
import json

# Register a global adapter for a custom type
class JsonAdapter:
    def __init__(self, value):
        self.value = value
    def getquoted(self):
        return psycopg2.extensions.QuotedString(
            json.dumps(self.value)
        ).getquoted()

psycopg2.extensions.register_adapter(dict, JsonAdapter)

# psycopg3 — scoped type adaptation (per-connection or per-cursor)
import psycopg
from psycopg.types.json import Jsonb

# Option 1: use the built-in Jsonb wrapper
conn.execute(
    "INSERT INTO events (data) VALUES (%s)",
    (Jsonb({"event": "click", "target": "button"}),)
)

# Option 2: register a custom dumper on the connection
class PointDumper(psycopg.adapt.Dumper):
    format = psycopg.pq.Format.TEXT
    def dump(self, obj):
        return f"({obj.x},{obj.y})".encode()

conn.adapters.register_dumper(Point, PointDumper)

The scoped approach is better engineering. Global mutable state is, as a rule, the source of bugs that appear only in production under concurrency. But if your codebase has a single adapters.py module that calls register_adapter() for twelve custom types at import time, you will need to refactor that into a function that runs against each new connection. This is straightforward but tedious, and the tedium scales with the number of custom types.

Error handling: named exceptions

psycopg3 introduces named exception classes for every PostgreSQL error code. Where psycopg2 required you to catch a generic IntegrityError and inspect e.pgcode to determine whether it was a unique violation, a foreign key violation, or a check constraint failure, psycopg3 gives you errors.UniqueViolation, errors.ForeignKeyViolation, and errors.CheckViolation as distinct exception types.

Error handling improvements
# psycopg3's improved error hierarchy
import psycopg
from psycopg import errors

try:
    conn.execute(
        "INSERT INTO users (email) VALUES (%s)", ("duplicate@example.com",)
    )
except errors.UniqueViolation as e:
    # Specific exception — no need to parse error codes
    print(f"Duplicate: {e.diag.constraint_name}")
    conn.rollback()
except errors.ForeignKeyViolation as e:
    # Every PostgreSQL error code has a named exception
    print(f"Missing reference: {e.diag.column_name}")
    conn.rollback()
except psycopg.OperationalError:
    # Connection-level failures
    pass

# psycopg2 required checking pgcode manually:
# except psycopg2.IntegrityError as e:
#     if e.pgcode == '23505':  # magic string
#         handle_duplicate()

This is not technically a breaking change — the old pattern of catching psycopg.IntegrityError still works. But it is worth updating during the migration, because the named exceptions make error handling code more readable and less dependent on magic strings.

Prepared statements: automatic and invisible

psycopg2 did not manage prepared statements. Every execution of the same SQL text went through the full cycle: parse, plan, execute. If you wanted prepared statements, you used cursor.execute() with the same query and hoped PostgreSQL's plan cache would help, but the driver itself did nothing to assist.

psycopg3 introduces automatic prepared statement management via the prepare_threshold parameter. After a query has been executed a configurable number of times (default: 5), the driver automatically prepares it on the server side. Subsequent executions skip the parse and plan phases entirely.

# psycopg3's automatic prepared statement management
import psycopg

conn = psycopg.connect(dsn)

# By default, prepare_threshold=5: after a query is executed 5 times
# with the same SQL text, psycopg3 automatically prepares it server-side.
# Subsequent executions use the prepared plan — no re-parsing, no re-planning.

for user_id in user_ids:
    conn.execute(
        "SELECT * FROM users WHERE id = %s", (user_id,)
    )
# First 5 calls: parse + plan + execute
# Call 6 onwards: execute only (prepared)

# Behind PgBouncer in transaction mode? Disable it:
conn = psycopg.connect(dsn, prepare_threshold=0)

# PgBouncer 1.21+ supports DEALLOCATE ALL, so you can also:
conn = psycopg.connect(dsn, prepare_threshold=5)
# Just ensure PgBouncer is configured with:
#   server_reset_query = DEALLOCATE ALL; DISCARD ALL

The performance impact depends on your workload. For applications that execute the same handful of queries thousands of times — which describes most web applications — the savings are meaningful: 10-30% reduction in query latency for prepared queries, because the server skips both parsing and planning.

The critical compatibility note: if you are behind PgBouncer in transaction mode (which is the most common PgBouncer configuration), prepared statements can cause InvalidSQLStatementName errors because PgBouncer rotates server connections between transactions, and the prepared statement exists only on the connection where it was created. Set prepare_threshold=0 to disable, or upgrade to PgBouncer 1.21+ which supports DEALLOCATE ALL as part of the server reset query.

The gevent trap: a 100x latency regression nobody warns you about

This is the section I wish someone had written before I learned it empirically.

If your application uses gevent — and a meaningful number of Python web applications do, particularly those running Gunicorn with gevent workers — psycopg3 can introduce a catastrophic latency regression. Not 2x. Not 5x. 100x.

# WARNING: psycopg3 + gevent = potential 100x latency regression
# See: https://github.com/psycopg/psycopg/issues/919
#
# psycopg3 uses the libpq nonblocking API by default.
# Under gevent's monkey-patching, the libpq polling loop
# can degrade to busy-waiting, causing:
#   - CPU spikes on idle connections
#   - 100x latency increase on simple queries
#   - Thread starvation in mixed workloads
#
# If you MUST use gevent with psycopg3:
# 1. Pin to psycopg[c] (C extension) — avoids pure-Python polling
# 2. Test latency under realistic concurrency before deploying
# 3. Consider whether gevent is still the right choice
#
# If you're using gunicorn + gevent workers:
# gunicorn --worker-class gevent app:app
# ^^^ Test this configuration thoroughly with psycopg3.
#
# The safest path: switch to uvicorn + asyncio if possible.

The root cause is documented in psycopg issue #919. psycopg3 uses libpq's nonblocking API and implements its own polling loop to wait for query results. Under gevent's monkey-patching, this polling loop can degrade to busy-waiting — consuming CPU while achieving nothing, starving other greenlets, and inflating query latency from microseconds to milliseconds.

The practical consequence: a query that takes 1ms with psycopg2 + gevent can take 100ms with psycopg3 + gevent. Under load, the busy-waiting compounds, threads starve, and the application becomes unresponsive.

Detecting the regression before it reaches production

The insidious quality of this bug is that it does not appear in development. A single connection executing queries sequentially will behave normally. The regression requires concurrent connections under gevent's cooperative scheduling to manifest — which is to say, it requires production-like conditions.

Diagnosing the gevent regression
# How to detect the gevent polling regression before it hits production
import time
import psycopg

def benchmark_query(dsn, iterations=1000):
    """Run under your actual server config (gunicorn + gevent workers)."""
    conn = psycopg.connect(dsn)
    latencies = []
    for _ in range(iterations):
        start = time.perf_counter()
        conn.execute("SELECT 1")
        latencies.append(time.perf_counter() - start)
    conn.close()

    avg_ms = sum(latencies) / len(latencies) * 1000
    p99_ms = sorted(latencies)[int(0.99 * len(latencies))] * 1000
    print(f"avg: {avg_ms:.2f}ms  p99: {p99_ms:.2f}ms")
    # Healthy psycopg3:  avg: 0.3ms  p99: 0.8ms
    # Gevent regression:  avg: 30ms   p99: 150ms
    # If your numbers look like the second line, you have the bug.

Run this benchmark under your actual deployment configuration. If your average latency is in the tens of milliseconds for a SELECT 1, the polling regression is present. If it is under 1ms, you are safe — likely because you are using the C extension, which uses a different polling mechanism.

Your options, in order of preference

  1. Switch from gevent to asyncio. psycopg3's async mode is first-class and does not have this issue. If you can migrate to uvicorn or hypercorn, do so. This is the correct long-term answer for most applications.
  2. Use the C extension. Install psycopg[c] rather than psycopg[binary]. The C implementation uses a different polling mechanism that is less susceptible to gevent interference. Note that psycopg[c] requires a C compiler and libpq headers at build time.
  3. Stay on psycopg2. If gevent is non-negotiable and the C extension does not resolve the issue in your environment, psycopg2 remains a perfectly viable driver. There is no shame in a working application.

Test under realistic concurrency before deploying. A simple SELECT 1 in development will not reveal this issue. You need concurrent queries under gevent workers to trigger the polling degradation.

The honest counterpoint on gevent itself

I should be forthcoming: the gevent compatibility issue is, in a sense, a symptom of a broader shift. Gevent's monkey-patching approach — intercepting system calls to make synchronous code appear asynchronous — was ingenious in 2012. In 2026, with native async/await in Python, native async support in every major web framework, and native async support in the database driver itself, monkey-patching is increasingly the wrong abstraction.

This is not to say your gevent application is wrong. It is to say that if you are investing the effort to migrate your database driver, it may be worth evaluating whether the concurrency model deserves the same consideration. A migration from gevent to asyncio alongside a migration from psycopg2 to psycopg3 is more work, but it resolves the compatibility issue permanently and positions the application on the runtime that the Python ecosystem is converging toward.

"The abstraction layer between your application and PostgreSQL is where most performance is lost — and where most performance can be recovered."

— from You Don't Need Redis, Chapter 3: The ORM Tax

How do Django and SQLAlchemy handle the transition?

The framework support is mature enough that the migration is largely a dependency swap — but there are configuration details worth knowing, and a few traps worth avoiding.

Django 4.2+

Django 4.2+ configuration
# settings.py — Django 4.2+ with psycopg3
DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        # Django 4.2+ auto-detects psycopg3 if installed.
        # No ENGINE change needed — same backend, new driver.

        "NAME": "mydb",
        "USER": "app",
        "HOST": "localhost",
        "PORT": "5432",

        # psycopg3-specific options
        "OPTIONS": {
            "pool": True,  # Django 5.1+ built-in pool support
        },
    }
}

# Verify which driver Django is using:
# python -c "import django.db.backends.postgresql; print(django.db.backends.postgresql.psycopg.__name__)"
# Output: "psycopg" (v3) or "psycopg2"

Django 4.2+ supports psycopg3 through the same django.db.backends.postgresql engine. No engine change is needed — Django detects which driver is installed and uses it accordingly. Django 5.1 added native connection pooling support via the "pool": True option, which uses psycopg_pool under the hood.

The migration path for Django: install psycopg[binary], uninstall psycopg2-binary, run your test suite. If you have custom database backends, type adapters, or raw SQL that uses psycopg2-specific features (like register_adapter), those will need updating.

Django migration gotchas
# Django gotchas when migrating to psycopg3

# 1. Custom database backends that import psycopg2 internals
# BAD — will break
from psycopg2.extensions import ISOLATION_LEVEL_SERIALIZABLE
# GOOD — use Django's API
from django.db import connection
connection.set_autocommit(False)

# 2. Raw SQL with register_adapter — needs updating
# psycopg2:
#   psycopg2.extensions.register_adapter(MyType, adapt_mytype)
# psycopg3 (in a custom backend or connection init):
#   connection.connection.adapters.register_dumper(MyType, MyTypeDumper)

# 3. The django-psycopg2-pool package — remove it
# psycopg_pool replaces it entirely. In Django 5.1+:
DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "OPTIONS": {
            "pool": {
                "min_size": 4,
                "max_size": 16,
                "timeout": 10,
            },
        },
    }
}

The most common Django migration failure I have seen is third-party packages that import psycopg2 directly. Run grep -r "psycopg2" your_venv/ after uninstalling psycopg2-binary to find any remaining references. Common offenders include older versions of django-environ, django-health-check, and various database backup utilities.

SQLAlchemy 2.0

SQLAlchemy 2.0 configuration
# SQLAlchemy 2.0 with psycopg3
from sqlalchemy import create_engine
from sqlalchemy.ext.asyncio import create_async_engine

# Sync — note the "psycopg" dialect (not "psycopg2")
engine = create_engine(
    "postgresql+psycopg://app:pass@localhost/mydb",
    pool_size=20,
    pool_pre_ping=True,
)

# Async — psycopg3's native async support
async_engine = create_async_engine(
    "postgresql+psycopg://app:pass@localhost/mydb",
    pool_size=20,
)

SQLAlchemy 2.0 ships a dedicated psycopg dialect. The connection URL changes from postgresql+psycopg2:// to postgresql+psycopg://. Critically, psycopg3's native async support means you can use create_async_engine without a greenlet-based shim — a genuine architectural improvement over the psycopg2 async story, which was always somewhat theatrical.

SQLAlchemy async without greenlet
# SQLAlchemy async with psycopg3 — no greenlet shim needed
from sqlalchemy.ext.asyncio import create_async_engine, async_sessionmaker

engine = create_async_engine(
    "postgresql+psycopg://app:pass@localhost/mydb",
    pool_size=20,
    pool_pre_ping=True,
)

SessionLocal = async_sessionmaker(engine, expire_on_commit=False)

async def get_active_users():
    async with SessionLocal() as session:
        result = await session.execute(
            select(User).where(User.active == True)
        )
        return result.scalars().all()

# With psycopg2, SQLAlchemy async required greenlet to bridge
# sync driver code into async contexts. This added:
#   - A dependency (greenlet)
#   - Context-switching overhead
#   - Confusing tracebacks when things went wrong
#
# With psycopg3, the async path is native. No shim. No overhead.
# The connection speaks async from the socket to your await.

The elimination of the greenlet dependency for async SQLAlchemy is, in my view, one of the strongest arguments for the migration in applications that use SQLAlchemy's async ORM. The greenlet shim worked, but it introduced a layer of abstraction that made debugging difficult, tracebacks confusing, and performance characteristics unpredictable. With psycopg3, the async path is native from the database socket to your await.

One gotcha with SQLAlchemy: if you are using pool_pre_ping=True (and you should be), psycopg3's connection health check is faster than psycopg2's because it uses the libpq ping mechanism rather than executing SELECT 1.

Flask

Flask does not have a built-in database integration to update, which means you have more flexibility — and more responsibility. If you were using Flask-SQLAlchemy, the SQLAlchemy migration applies. If you were managing connections directly, here is the pattern I recommend:

Flask with psycopg_pool
# Flask with psycopg3 — using psycopg_pool directly
from flask import Flask, g
from psycopg_pool import ConnectionPool
from psycopg.rows import dict_row

pool = ConnectionPool(
    "dbname=mydb user=app host=localhost",
    min_size=4,
    max_size=16,
    kwargs={"row_factory": dict_row},
)

app = Flask(__name__)

@app.before_request
def get_conn():
    g.conn = pool.getconn()

@app.teardown_request
def put_conn(exc):
    conn = g.pop("conn", None)
    if conn is not None:
        if exc:
            conn.rollback()
        pool.putconn(conn)

@app.route("/users")
def list_users():
    rows = g.conn.execute("SELECT id, email FROM users").fetchall()
    return {"users": rows}

The key improvement over the psycopg2 equivalent is the pool itself. With psycopg2, you were either using ThreadedConnectionPool (inadequate) or PgBouncer (external dependency). With psycopg_pool, you get production-grade pooling in process, with health checks and recycling, and it integrates naturally with Flask's request lifecycle.

The psycopg[binary] vs psycopg[c] decision

When you pip install psycopg, you get the pure-Python implementation. It works everywhere Python runs, but it is the slowest option. For production, you want one of the accelerated variants, and the choice between them matters more than the documentation suggests.

psycopg[binary] ships a pre-compiled libpq bundled with the package. No system dependencies, no C compiler, no pg_config needed. Install it and it works. This is the path of least resistance, and for most applications, it is the correct choice.

psycopg[c] compiles a C extension against your system's libpq. It requires libpq-dev (or equivalent) and a C compiler at install time. In exchange, it uses your system's libpq — which means it inherits your system's SSL configuration, Kerberos support, and any patches your distribution applies. It also uses a different internal polling mechanism, which is relevant for the gevent issue discussed earlier.

The performance difference between psycopg[binary] and psycopg[c] is minimal — both use C-level acceleration for the hot paths. The difference is operational: psycopg[binary] is easier to install but bundles its own libpq, while psycopg[c] is harder to install but uses your system's libpq.

For Docker-based deployments, psycopg[binary] is usually correct — fewer build dependencies in your Dockerfile. For system-level deployments where libpq is already installed and managed by the OS package manager, psycopg[c] is preferable because it stays in sync with system library updates.

The honest case against migrating

A waiter who overstates his case is no waiter at all. There are legitimate reasons to defer this migration, and I would be doing you a disservice to omit them.

Your application is stable and performance is acceptable. psycopg2 is not broken. If your application handles its current load comfortably, your queries are well-indexed, and your connection management is not causing issues, the migration provides optimization rather than correction. Optimization is valuable, but it is not urgent.

You have extensive custom type adapters. If your codebase has a sophisticated type adaptation layer — custom JSON handling, PostGIS types, custom composite types, application-specific serialization — the migration of those adapters from psycopg2's global registry to psycopg3's scoped system is the most labor-intensive part of the migration. For a handful of adapters, it is an afternoon. For dozens, it is a project.

You depend on gevent and cannot switch to asyncio. If your application is deeply committed to gevent — gevent workers, gevent-aware libraries, gevent-patched everything — the polling regression described above is a genuine blocker. The C extension mitigates it in most cases, but "most cases" is not a guarantee, and testing under production concurrency is not optional.

Third-party dependencies that require psycopg2. Some packages explicitly import from psycopg2 rather than using database-agnostic APIs. If a critical dependency has not been updated for psycopg3, you will need to either fork it, find an alternative, or wait. This is rarer in 2026 than it was in 2023, but it still occurs.

If any of these apply to your situation, a deferred migration is not a failure — it is a prioritization. psycopg2 continues to receive maintenance updates. Your application will not stop working.

A migration sequence that minimizes risk

Having done this several times, I would suggest the following order of operations:

  1. Install both drivers simultaneously. psycopg2-binary and psycopg[binary] can coexist — different package names, different import paths. This lets you migrate incrementally.
  2. Audit your codebase for psycopg2-specific usage. Search for import psycopg2, from psycopg2, register_adapter, RealDictCursor, copy_from, copy_to, and cursor(name=. Each of these is a migration point. Count them. This count is your scope estimate.
  3. Run your test suite against psycopg3. Swap the import in a test configuration and run everything. The failures will map directly to the breaking changes table above.
  4. Fix the breaking changes. Most are mechanical: import paths, cursor factories, COPY API. Budget a day for a mid-sized codebase. If you have custom type adapters, budget two.
  5. Replace the connection pool. If you were using ThreadedConnectionPool, switch to psycopg_pool.ConnectionPool. If you were using an external pooler, test psycopg3's prepare_threshold setting against your pooler configuration.
  6. Enable binary protocol selectively. Start with read-heavy endpoints that return large result sets. Measure before and after. The gains are most pronounced on queries returning hundreds or thousands of rows with numeric types.
  7. Test under gevent if applicable. This is not optional. See the section above.
  8. Deploy behind a feature flag. Route a percentage of traffic to the psycopg3 code path. Monitor latency distributions, error rates, and connection pool metrics. Increase the percentage as confidence builds.
  9. Adopt idiomatic patterns gradually. Once the migration is stable, begin refactoring toward psycopg3 idioms — context managers, conn.execute(), pipeline mode where appropriate. This is not part of the migration itself, but it is where the long-term code quality improvements live.

The entire migration for a Django application with 50 models and no custom type adapters took me about four hours, including testing. An application with extensive raw SQL, custom adapters, and COPY-based ETL pipelines took closer to two days. Neither required any schema changes or database-side modifications.

I should note that the feature flag approach in step 8 is not always practical. If your application has a single database connection configuration — one pool, one DSN — running both drivers simultaneously requires architectural changes that may not be worth the effort. For simpler applications, steps 1 through 7 followed by a direct cutover during a maintenance window is entirely reasonable. The feature flag is for applications where the stakes of a regression justify the additional complexity.

Common mistakes I have observed during migration

Patterns that appear in virtually every psycopg2-to-psycopg3 migration I have reviewed. Consider this a pre-flight checklist.

Forgetting to install psycopg_pool separately. Unlike psycopg2, where pooling (such as it was) came bundled, psycopg3's pool is a separate package. pip install psycopg[binary] does not include it. You need pip install psycopg_pool explicitly, or pip install "psycopg[binary,pool]" to get both.

Using conn.cursor() everywhere instead of conn.execute(). psycopg3 still supports creating cursors explicitly, and a mechanical migration will preserve this pattern. But conn.execute() is not just a shorthand — it creates and manages the cursor internally, which means one fewer object to close, one fewer resource to leak, and one fewer line of code to read.

Not updating prepare_threshold for PgBouncer. If you run PgBouncer in transaction mode and do not set prepare_threshold=0, psycopg3 will begin preparing statements after the fifth execution of any given query. When PgBouncer rotates the server connection, the prepared statement no longer exists, and you receive an InvalidSQLStatementName error. This error appears intermittently — only when PgBouncer assigns a different server connection — which makes it particularly unpleasant to diagnose.

Treating the migration as purely mechanical. A find-and-replace from psycopg2 to psycopg will get you a working application. It will not get you a well-migrated one. The context managers, the row_factory system, the pipeline mode, the scoped type adaptation — these are design improvements, not just API changes. A migration that preserves every psycopg2 pattern in psycopg3 syntax is a missed opportunity.

The driver is the last 5% of your performance story

I have spent considerable time on driver migration, and I want to be direct about something: switching from psycopg2 to psycopg3 is worth doing. The binary protocol alone can halve your data transfer overhead. The pooling is dramatically better. The async support is genuine rather than bolted on. Pipeline mode saves real latency on real workloads.

But the driver is the last 5% of your PostgreSQL performance story. The first 95% is whether your queries hit indexes, whether your joins are efficient, whether you are fetching 10,000 rows to display 10, and whether that GROUP BY on a 50-million-row table is materializing a view or recomputing from scratch on every request.

A query that performs a sequential scan on a 10-million-row table takes 4,000ms. The binary protocol saves perhaps 15ms of that. You have optimized the wrong layer.

I have seen teams invest a week migrating from psycopg2 to psycopg3 and celebrate a 40% throughput improvement on their data export pipeline — only to discover that the same pipeline was doing a full table scan because an index was missing. Adding the index reduced the query time from 4,000ms to 3ms. The binary protocol then saved an additional 0.5ms. Both improvements are real. One is 8,000 times larger than the other.

Gold Lapel sits between your application and PostgreSQL — regardless of whether you are using psycopg2, psycopg3, or asyncpg — and addresses the queries themselves. Missing indexes created automatically. Expensive aggregations materialized. N+1 patterns detected and rewritten. The 4,000ms query becomes 3ms, and the driver choice becomes the pleasant optimization it should be rather than a desperate measure.

Migrate to psycopg3 for the pooling, the binary protocol, the pipeline mode, and the modern API. Then attend to the queries. That is where the real performance lives.

Frequently asked questions

Terms referenced in this article

Allow me to suggest a companion piece. I have written at some length on the performance differences between asyncpg and psycopg3 under FastAPI — a natural next question once you have completed the migration and are considering your async options.