Skip to content

sqlite3.Connection.backup() blocks indefinitely on a locked source DB; connect(timeout) does not apply #162414

Open

Description

@daneeller

Environment

  • CPython 3.13.14 (Windows 11), sqlite3 module. The behavior also appears in earlier CPython versions; I only runtime-tested 3.13.

What happens

sqlite3.Connection.backup() can block indefinitely when the source database is locked by another connection. The timeout argument of sqlite3.connect() does not bound this wait — it applies to locked table operations, not to the backup retry loop.

In the repro below, a parent process holds BEGIN EXCLUSIVE on the source database. The child calls src.backup(dst) and never returns while the lock is held (killed after 4 seconds). Once the lock is released, the identical child code finishes in 0.06 s.

Repro

# parent.py: hold an exclusive write lock for 4 seconds, then release
import sqlite3, subprocess, sys, time

CHILD = """
import sqlite3, sys
src = sqlite3.connect(sys.argv[1], timeout=1)
dst = sqlite3.connect(sys.argv[2], timeout=1)
print('child: calling backup()', flush=True)
src.backup(dst)
print('child: backup() returned', flush=True)
"""

conn = sqlite3.connect("locked.db")
conn.execute("CREATE TABLE t(x)")
conn.commit()
conn.execute("BEGIN EXCLUSIVE")
conn.execute("INSERT INTO t VALUES (1)")

p = subprocess.Popen([sys.executable, "-c", CHILD, "locked.db", "backup.db"],
                     stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
try:
    p.communicate(timeout=4)
except subprocess.TimeoutExpired:
    p.kill()
    p.communicate()
    print("child was still blocked after 4 s")

conn.rollback()
conn.close()

Observed:

child: calling backup()
child was still blocked after 4 s
child: calling backup()      # control run, lock released
child: backup() returned

Documentation gap

The current backup() docs say:

Works even if the database is being accessed by other clients or concurrently by the same connection.

and define sleep as:

The number of seconds to sleep between successive attempts to back up remaining pages.

They do not say that backup() retries on SQLITE_BUSY/SQLITE_LOCKED until it succeeds or a real error occurs, nor that connect(timeout=...) has no effect on this wait. A reader can reasonably assume the connection timeout applies, which is not the case.

Suggested documentation change

Add a note to the sqlite3.Connection.backup() section, e.g.:

backup() retries internally when the source database is locked; the timeout passed to connect() does not limit this wait. To bound the total time, pass a progress callback that checks your own deadline and raises, or call backup() with a bounded number of pages per step.

If a code change is considered instead, an optional deadline/timeout parameter on backup() would be useful, but I understand that is an API change and may need broader discussion.

Impact

A background backup thread that relies on connect(timeout=...) can hang forever if another process holds a write lock, with no exception and no timeout to trigger cleanup or retry logic.

No project-specific or private data is included in this report; the repro uses only the standard library.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions