Chapter 08

Fixture scope and conftest.py — sharing setup safely

Build slow setup once and share it: the five scopes, --setup-show, the danger of shared state and ScopeMismatch, conftest.py for fixtures without imports and per-directory overrides, what autouse costs, and the order fixtures are created in.

50 minPython 3.12
  1. 1Encounter
  2. 2Understand
  3. 3Worked
  4. 4Predict
  5. 5Apply
  6. 6Stretch

The problem we are solving

Last chapter's fixtures were built fresh for every test. That is the right default — until the thing being built is expensive. Here is a database connection that takes half a second to open, the way a real server handshake might:

test_users.py:

python
import sqlite3
import time

import pytest


def connect():
    time.sleep(0.5)  # stands in for a slow server handshake
    conn = sqlite3.connect(":memory:")
    conn.execute("CREATE TABLE users (name TEXT)")
    return conn


@pytest.fixture
def db():
    conn = connect()
    yield conn
    conn.close()


def test_starts_empty(db):
    count = db.execute("SELECT COUNT(*) FROM users").fetchone()[0]
    assert count == 0


def test_insert_one(db):
    db.execute("INSERT INTO users VALUES ('asha')")
    count = db.execute("SELECT COUNT(*) FROM users").fetchone()[0]
    assert count == 1


def test_names_are_text(db):
    db.execute("INSERT INTO users VALUES ('ravi')")
    name = db.execute("SELECT name FROM users").fetchone()[0]
    assert name == "ravi"

pytest -q --durations=3 reports where the time went:

text
...                                                                      [100%]
============================= slowest 3 durations ==============================
0.50s setup    test_users.py::test_starts_empty
0.50s setup    test_users.py::test_insert_one
0.50s setup    test_users.py::test_names_are_text
3 passed in 1.65s

Three tests, three connections, a second and a half spent shaking hands. With three hundred tests that is two and a half minutes of nothing, and a suite that slow stops being run.

The question this chapter answers is: how do you build something once and share it — without letting the tests that share it trip over each other?

By the end of this chapter you can

  • Choose a fixture's scope: function, class, module, package or session
  • Watch fixtures being set up and torn down with --setup-show
  • Explain why a wide-scoped fixture holding mutable state makes tests depend on each other — and fix it
  • Read a ScopeMismatch error and know which fixture to change
  • Share fixtures through conftest.py, and override one for a single directory
  • Say what autouse=True costs, and list every fixture available with --fixtures
  • Predict the order in which fixtures are created

Prerequisites: fixtures.


Before you write the test

Scope is not something you add at the end to make a slow suite fast. It is decided before the first test is written, from two questions about every piece of setup: how much does it cost to build, and does a test change it? Answer those first, and the right scope follows.

The contract. The code under test here is a small users table. What it promises: a new database has no users; inserting a user adds exactly one row; the name you insert is the name you get back. Each of those promises must hold no matter which tests ran before. That last clause is the one this chapter is really about.

What you must have in place. A virtual environment with pytest installed; sqlite3, which ships with Python, so there is nothing to install; the code under test importable from the test files (here, files side by side in one directory, run from that directory); and a decision about where shared fixtures will live — a conftest.py at the top of the test tree.

The test plan. Three cases, each of which must pass alone and in any order:

| case | input | expected | |---|---|---| | fresh database | nothing inserted | COUNT(*) is 0 | | one insert | 'asha' | COUNT(*) is 1 | | name round-trip | 'ravi' | the first name read back is 'ravi' |

The setup plan. Then the same kind of table for what the tests stand on:

| resource | cost to build | does a test change it? | scope | |---|---|---|---| | connection + table | slow (half a second) | no | wide — module or session | | the rows in the table | free | yes, every insert | function — reset after each test |

Read the last column carefully: one object, the connection, holds both a slow thing and a changing thing. That is why the chapter ends up with two fixtures — one wide, one narrow — instead of one.

What not to test. Not sqlite3 itself; it is tested far more thoroughly than your code will ever be. Not the fixtures directly — a fixture is checked by the tests that use it, and by reading --setup-show. And not the half-second delay: speed is something you measure with --durations, not something you assert.


scope= — once per what?

A fixture's scope says how long one value lives before pytest throws it away and builds another. The default is function: one per test. Change one line:

python
@pytest.fixture(scope="module")
def db():
    conn = connect()
    yield conn
    conn.close()

pytest -q:

text
..F                                                                      [100%]
=================================== FAILURES ===================================
_____________________________ test_names_are_text ______________________________

db = <sqlite3.Connection object at 0x79350aa45c60>

    def test_names_are_text(db):
        db.execute("INSERT INTO users VALUES ('ravi')")
        name = db.execute("SELECT name FROM users").fetchone()[0]
>       assert name == "ravi"
E       AssertionError: assert 'asha' == 'ravi'
E
E         - ravi
E         + asha

test_users.py:35: AssertionError
=========================== short test summary info ============================
FAILED test_users.py::test_names_are_text - AssertionError: assert 'asha' == ...
1 failed, 2 passed in 0.52s

Two things happened. The run went from 1.65 seconds to 0.52 — one connection instead of three. And a test that passed a moment ago now fails. Keep that failure in mind; it gets its own section below. First, the scopes themselves.

There are five, from narrowest to widest:

| scope | one value per | torn down after | |---|---|---| | function | test (the default) | each test | | class | test class | the last test in the class | | module | test file | the last test in the file | | package | directory where the fixture is defined | the last test in that directory, sub-directories included | | session | whole pytest run | the last test of the run |

A counter makes the difference visible. Each fixture below counts how many times it was built:

test_scopes.py:

python
from collections import Counter

import pytest

made = Counter()


@pytest.fixture(scope="session")
def per_session():
    made["session"] += 1


@pytest.fixture(scope="module")
def per_module():
    made["module"] += 1


@pytest.fixture(scope="class")
def per_class():
    made["class"] += 1


@pytest.fixture
def per_test():
    made["function"] += 1


class TestFirst:
    def test_a(self, per_session, per_module, per_class, per_test):
        pass

    def test_b(self, per_session, per_module, per_class, per_test):
        pass


class TestSecond:
    def test_c(self, per_session, per_module, per_class, per_test):
        pass


def test_report():
    print(dict(made))

pytest -q -s (the -s lets print through):

text
...{'session': 1, 'module': 1, 'class': 2, 'function': 3}
.
4 passed in 0.01s

Three tests used the fixtures. The function fixture was built three times, the class fixture twice (two classes), and the module and session fixtures once each.

Watching it happen: --setup-show

Counting works, but pytest can draw the whole timeline for you. Run the same file with pytest -q --setup-show:

text
SETUP    S per_session
    SETUP    M per_module
      SETUP    C per_class
        SETUP    F per_test
        test_scopes.py::TestFirst::test_a (fixtures used: per_class, per_module, per_session, per_test) .
        TEARDOWN F per_test
        SETUP    F per_test
        test_scopes.py::TestFirst::test_b (fixtures used: per_class, per_module, per_session, per_test) .
        TEARDOWN F per_test
      TEARDOWN C per_class
      SETUP    C per_class
        SETUP    F per_test
        test_scopes.py::TestSecond::test_c (fixtures used: per_class, per_module, per_session, per_test) .
        TEARDOWN F per_test
      TEARDOWN C per_class
        test_scopes.py::test_report .
    TEARDOWN M per_module
TEARDOWN S per_session
4 passed in 0.01s

The letter after SETUP is the scope — Session, Package, Module, Class, Function — and the indentation nests them. Read it top to bottom and you can see exactly when each value is born and when it dies. Whenever you are unsure what a fixture is doing, this is the first flag to reach for.

A shared fixture shares its state

Back to the failure. With scope="module" all three tests received the same connection. test_insert_one added 'asha' and nothing removed her. Then test_names_are_text added 'ravi', asked for the first name, and got 'asha'.

The clearest sign of this bug is a test that passes on its own. pytest -q test_users.py::test_names_are_text:

text
.                                                                        [100%]
1 passed in 0.50s

Alone it passes; in company it fails. Tests that depend on what ran before them are worse than slow tests: they fail in one order and pass in another, and they waste an afternoon each time.

The rule: widen the scope of the expensive thing, keep the scope of the state narrow. Split the fixture in two — a wide one that owns the connection, and a function-scoped one that cleans up after each test:

python
@pytest.fixture(scope="module")
def connection():
    conn = connect()
    yield conn
    conn.close()


@pytest.fixture
def db(connection):
    yield connection
    connection.rollback()  # undo whatever this test wrote

The tests do not change — they still ask for db. pytest -q --durations=3:

text
...                                                                      [100%]
============================= slowest 3 durations ==============================
0.50s setup    test_users.py::test_starts_empty

(2 durations < 0.005s hidden.  Use -vv to show these durations.)
3 passed in 0.51s

One connection, three passing tests. sqlite3 opens a transaction on the first INSERT, and rollback() throws the whole transaction away, so every test starts with an empty table.

Why put the cleanup in a fixture, rather than starting each test with DELETE FROM users? Because a cleanup that every test has to remember is a cleanup that one test will forget — and the test that forgets is never the one that fails; the next one is. In the fixture, the cleanup happens after every test that asks for db, including the tests written next year by someone who has never read this file. And why rollback() rather than DELETE? A rollback undoes everything the test wrote, in every table, without the fixture having to know what those writes were.

--setup-show shows the split:

text
SETUP    M connection
        SETUP    F db (fixtures used: connection)
        test_users.py::test_starts_empty (fixtures used: connection, db) .
        TEARDOWN F db
        SETUP    F db (fixtures used: connection)
        test_users.py::test_insert_one (fixtures used: connection, db) .
        TEARDOWN F db
        SETUP    F db (fixtures used: connection)
        test_users.py::test_names_are_text (fixtures used: connection, db) .
        TEARDOWN F db
    TEARDOWN M connection
3 passed in 0.51s

A function-scoped fixture may freely request a wider one, as db requests connection here. The opposite direction is not allowed.

ScopeMismatch — wide may not ask for narrow

test_mismatch.py:

python
import sqlite3

import pytest


@pytest.fixture
def db_name():
    return ":memory:"


@pytest.fixture(scope="session")
def connection(db_name):
    conn = sqlite3.connect(db_name)
    yield conn
    conn.close()


def test_connects(connection):
    assert connection.execute("SELECT 1").fetchone() == (1,)

pytest -q:

text
E                                                                        [100%]
==================================== ERRORS ====================================
_______________________ ERROR at setup of test_connects ________________________
ScopeMismatch: You tried to access the function scoped fixture db_name with a session scoped request object. Requesting fixture stack:
test_mismatch.py:11:  def connection(db_name)
Requested fixture:
test_mismatch.py:6:  def db_name()
=========================== short test summary info ============================
ERROR test_mismatch.py::test_connects - Failed: ScopeMismatch: You tried to a...
1 error in 0.01s

Think about what the request would mean. connection lives for the whole run; db_name is thrown away after the first test. The session value would be holding on to something that, by its own rules, no longer exists. So pytest refuses — and note that it is an Error at setup, not a Failure: the test never ran.

The message names both fixtures and both lines. The fix is always the same: make the requested fixture at least as wide as the one asking for it — here @pytest.fixture(scope="session") on db_name — or make the asking fixture narrower.

conftest.py — fixtures without imports

Once a second test file needs db, copying the fixture is the wrong answer. Move it into a file named exactly conftest.py. Pytest loads that file by itself, and every test in the same directory and below it can request its fixtures — with no import.

A project with a sub-directory:

text
conftest.py
test_users.py
reports/
    conftest.py
    test_reports.py

conftest.py:

python
import sqlite3
import time

import pytest


def connect():
    time.sleep(0.5)  # stands in for a slow server handshake
    conn = sqlite3.connect(":memory:")
    conn.execute("CREATE TABLE users (name TEXT, city TEXT)")
    return conn


@pytest.fixture(scope="session")
def connection():
    """One database connection, shared by the whole run."""
    conn = connect()
    yield conn
    conn.close()


@pytest.fixture
def db(connection):
    """The shared connection; every write is rolled back after the test."""
    yield connection
    connection.rollback()

test_users.py — no imports at all:

python
def test_starts_empty(db):
    count = db.execute("SELECT COUNT(*) FROM users").fetchone()[0]
    assert count == 0


def test_insert_one(db):
    db.execute("INSERT INTO users VALUES ('asha', 'Dhaka')")
    count = db.execute("SELECT COUNT(*) FROM users").fetchone()[0]
    assert count == 1

The scope is now session: one connection for every file in the project.

Overriding a fixture for one directory

The tests in reports/ all want a database that already has users in it. A conftest.py in that directory can define a fixture with the same name; for tests in reports/ the nearer definition wins. And if it requests its own name, it receives the parent's version:

reports/conftest.py:

python
import pytest


@pytest.fixture
def db(db):
    """The parent db, with three users already in it."""
    db.executemany(
        "INSERT INTO users VALUES (?, ?)",
        [("asha", "Dhaka"), ("ravi", "Pune"), ("omar", "Cairo")],
    )
    return db

reports/test_reports.py:

python
def test_user_count(db):
    count = db.execute("SELECT COUNT(*) FROM users").fetchone()[0]
    assert count == 3


def test_cities_sorted(db):
    rows = db.execute("SELECT city FROM users ORDER BY city").fetchall()
    assert [city for (city,) in rows] == ["Cairo", "Dhaka", "Pune"]

pytest -v:

text
============================= test session starts ==============================
collecting ... collected 4 items

reports/test_reports.py::test_user_count PASSED                          [ 25%]
reports/test_reports.py::test_cities_sorted PASSED                       [ 50%]
test_users.py::test_starts_empty PASSED                                  [ 75%]
test_users.py::test_insert_one PASSED                                    [100%]

============================== 4 passed in 0.51s ===============================

Look at the order. The reports/ tests ran first and inserted three users each time — yet test_starts_empty, which ran after them, still found an empty table. The parent db rolled back the seeded rows too, because the override is built on top of it. --setup-show shows the two layers, both named db:

text
SETUP    S connection
        SETUP    F db (fixtures used: connection)
        SETUP    F db (fixtures used: db)
        reports/test_reports.py::test_user_count (fixtures used: connection, db) .
        TEARDOWN F db
        TEARDOWN F db

(That is the first test; the rest repeat the pattern.) The override only reaches downward: tests in the top directory never see it.

--fixtures — what can I ask for here?

With fixtures spread over several conftest.py files, you need a way to see what a given test can request. pytest --fixtures reports lists everything visible from that directory. The list starts with pytest's built-in fixtures (chapter nine) and those of installed plugins; yours come at the end:

text
------------------------ fixtures defined from conftest ------------------------
connection [session scope] -- conftest.py:15
    One database connection, shared by the whole run.

db -- conftest.py:23
    The shared connection; every write is rolled back after the test.

db -- reports/conftest.py:5
    The parent db, with three users already in it.

Each entry gives the name, the scope (when it is not function), the file and line, and the first lines of the docstring. That last part is the reason to write a one-line docstring on every shared fixture: it is the documentation people will actually read.

autouse=True — and what it costs

A fixture marked autouse=True is used by every test in its reach, whether the test asks for it or not. Added to the top conftest.py, this one checks that no test leaves rows behind:

python
@pytest.fixture(autouse=True)
def no_rows_left(connection):
    """After every test, check that the users table is empty again."""
    yield
    count = connection.execute("SELECT COUNT(*) FROM users").fetchone()[0]
    assert count == 0, f"{count} row(s) left behind"

It works — a test that writes through connection directly and skips the rollback is caught at teardown. But now add a test that has nothing to do with databases:

test_text.py:

python
def slugify(title):
    return title.strip().lower().replace(" ", "-")


def test_slugify():
    assert slugify(" Hello World ") == "hello-world"

pytest -q --setup-show test_text.py:

text
SETUP    S connection
        SETUP    F no_rows_left (fixtures used: connection)
        test_text.py::test_slugify (fixtures used: connection, no_rows_left) .
        TEARDOWN F no_rows_left
TEARDOWN S connection
1 passed in 0.50s

A one-line string test now opens a database and takes half a second. That is the cost of autouse: it is invisible — nothing in test_slugify says a database is involved — and it is unconditional — it runs for every test below its conftest.py, including the ones that do not need it. Keep autouse for things that are cheap and truly universal, and put it in the narrowest conftest.py that covers the tests that need it.

Which fixture is created first?

When a test needs several fixtures, pytest orders them by three rules:

  1. Wider scope first. Session before module before function, whatever the order in the test's signature.
  2. Within a scope, autouse fixtures first.
  3. Dependencies before the fixtures that need them.

test_order.py:

python
import pytest

log = []


@pytest.fixture(scope="session")
def server():
    log.append("server")


@pytest.fixture(scope="module")
def connection(server):
    log.append("connection")


@pytest.fixture
def user():
    log.append("user")


@pytest.fixture
def cart(user):
    log.append("cart")


@pytest.fixture(autouse=True)
def audit():
    log.append("audit")


def test_order(cart, connection):
    print(log)

pytest -q -s:

text
['server', 'connection', 'audit', 'user', 'cart']
.
1 passed in 0.01s

The test asked for cart before connection, but connection — module scope — was built first, after its own dependency server. Among the function-scoped fixtures, the autouse audit came first, then user, which cart needs. Teardown runs in exactly the reverse order. Beyond these rules, do not rely on order: if one fixture must exist before another, make it a dependency by naming it as a parameter.


A complete example

A small shop module, tested against one shared in-memory database.

shop.py:

python
def add_order(conn, customer, total):
    conn.execute(
        "INSERT INTO orders (customer, total) VALUES (?, ?)", (customer, total)
    )


def revenue(conn):
    return conn.execute("SELECT COALESCE(SUM(total), 0) FROM orders").fetchone()[0]


def top_customer(conn):
    row = conn.execute(
        "SELECT customer FROM orders GROUP BY customer "
        "ORDER BY SUM(total) DESC LIMIT 1"
    ).fetchone()
    return row[0] if row else None

conftest.py:

python
import sqlite3
import time

import pytest


@pytest.fixture(scope="session")
def connection():
    """One in-memory database with the orders table, for the whole run."""
    time.sleep(0.5)  # stands in for a slow server handshake
    conn = sqlite3.connect(":memory:")
    conn.execute("CREATE TABLE orders (customer TEXT, total REAL)")
    yield conn
    conn.close()


@pytest.fixture
def db(connection):
    """The shared connection; whatever a test writes is rolled back."""
    yield connection
    connection.rollback()

test_orders.py:

python
from shop import add_order, revenue, top_customer


def test_revenue_starts_at_zero(db):
    assert revenue(db) == 0


def test_add_order_counts_toward_revenue(db):
    add_order(db, "asha", 250.0)
    assert revenue(db) == 250.0


def test_no_top_customer_without_orders(db):
    assert top_customer(db) is None

reports/conftest.py:

python
import pytest

from shop import add_order


@pytest.fixture
def db(db):
    """The parent db, with four orders from three customers already in it."""
    add_order(db, "asha", 250.0)
    add_order(db, "ravi", 900.0)
    add_order(db, "omar", 120.0)
    add_order(db, "asha", 700.0)
    return db

reports/test_reports.py:

python
from shop import revenue, top_customer


def test_total_revenue(db):
    assert revenue(db) == 1970.0


def test_top_customer_by_total(db):
    assert top_customer(db) == "asha"

pytest -v --durations=1:

text
============================= test session starts ==============================
collecting ... collected 5 items

reports/test_reports.py::test_total_revenue PASSED                       [ 20%]
reports/test_reports.py::test_top_customer_by_total PASSED               [ 40%]
test_orders.py::test_revenue_starts_at_zero PASSED                       [ 60%]
test_orders.py::test_add_order_counts_toward_revenue PASSED              [ 80%]
test_orders.py::test_no_top_customer_without_orders PASSED               [100%]

============================= slowest 1 durations ==============================
0.50s setup    reports/test_reports.py::test_total_revenue
============================== 5 passed in 0.51s ===============================

Three things are worth noticing.

First, the slow setup happened once, on the first test of the run, and nowhere else. That is the session scope paying off.

Second, the tests in test_orders.py ran after the report tests had inserted eight orders in total, and still saw zero revenue. Replace connection.rollback() with pass and run again, and the price of shared state shows up immediately:

text
FAILED test_orders.py::test_revenue_starts_at_zero - assert 3940.0 == 0
FAILED test_orders.py::test_add_order_counts_toward_revenue - assert 4190.0 =...
FAILED test_orders.py::test_no_top_customer_without_orders - AssertionError: ...
3 failed, 2 passed in 0.52s

Third, no test file imports a fixture. test_orders.py imports shop, the code under test, and nothing else; the fixtures arrive by name from the nearest conftest.py.


When it breaks

ScopeMismatch: You tried to access the function scoped fixture db_name with a session scoped request object. A wide fixture requested a narrower one. Widen the requested fixture (db_name here) to at least the scope of the one asking, or narrow the one asking. The two lines under Requesting fixture stack tell you which is which.

AssertionError: assert 'asha' == 'ravi' — but the test passes when run on its own State is leaking between tests through a fixture wider than function. Run with --setup-show to see which fixture is shared, then split it: keep the expensive object wide and add a function-scoped fixture that undoes each test's changes.

fixture 'report_title' not found The fixture exists, but not where this test can see it. A conftest.py only serves its own directory and the directories below it — a fixture in reports/conftest.py is invisible to a test in the directory above. Move it up to a conftest.py that covers both. pytest --fixtures path/to/test_file.py shows exactly what that file can see.

Rows from one test appear in another even though db rolls back Something called commit(). A rollback only discards what has not been committed, so code that commits — or a test that uses connection directly instead of db — writes permanently. A guard like the no_rows_left fixture turns that into a visible ERROR at teardown ... AssertionError: 1 row(s) left behind.

A test that touches no database has become slow An autouse fixture somewhere above it depends on the expensive one. pytest --setup-show on that test lists every fixture it really used, including the ones it never asked for.

from conftest import db in a test file Do not import from conftest.py. Pytest already gives you its fixtures by name, and import conftest gets whichever conftest.py Python happens to find first. In the project above, adding that line to the top-level test_users.py imported the one in reports/ — the seeding override — and test_starts_empty failed with assert 3 == 0. An imported fixture also counts as defined in the test file itself, which beats every conftest.py. Request fixtures by naming them as parameters, and nothing else.