Fixtures — setup written once

The problem we are solving

Here is a small shopping cart, cart.py:

class Cart:
    def __init__(self):
        self.lines = []

    def add(self, name, price, quantity=1):
        if quantity < 1:
            raise ValueError(f"quantity must be at least 1: {quantity}")
        self.lines.append((name, price, quantity))

    def total(self):
        return sum(price * quantity for _, price, quantity in self.lines)

    def count(self):
        return sum(quantity for _, _, quantity in self.lines)

And three tests for it, test_cart.py:

from cart import Cart


def test_total():
    cart = Cart()
    cart.add("pen", 15.0, 3)
    cart.add("bag", 850.0)
    assert cart.total() == 895.0


def test_count():
    cart = Cart()
    cart.add("pen", 15.0, 3)
    cart.add("bag", 850.0)
    assert cart.count() == 4


def test_add_one_more():
    cart = Cart()
    cart.add("pen", 15.0, 3)
    cart.add("bag", 850.0)
    cart.add("notebook", 60.0, 2)
    assert cart.total() == 1015.0

pytest -q
...                                                                      [100%]
3 passed in 0.01s

They pass, and they are tiresome to read. Three tests, nine lines of setup, three lines that actually check something. The setup is the same in each, so the one line that differs — the line the test exists for — is buried.

It also costs something later. When Cart() starts needing an argument, you will change it in three places, or thirty. Last chapter, parametrize removed repetition in the inputs of a test. This chapter removes repetition in its setup.

By the end of this chapter you can

• Write a fixture with @pytest.fixture and request it by naming it as a test parameter
• Explain why each test gets its own fresh copy, and show it with a list
• Build one fixture out of another
• Write a yield fixture that cleans up after itself, and predict the order of setup and teardown
• Watch fixtures being created with --setup-show
• Read fixture 'x' not found and the "called directly" error
• Choose between a fixture and a plain helper function

Prerequisites: running one test over many cases with `parametrize`.

Before you write the test

Fixtures are not something you add to tests afterwards. They fall out of a plan, and the plan comes first.

What is promised. Cart promises three things: add() records a line and refuses a quantity below 1 with ValueError; total() is the sum of price × quantity; count() is the sum of the quantities. No files, no network, no global state — so far nothing that needs undoing.

What must be in place. The same as every chapter: a virtual environment with pytest installed, and cart.py importable from the folder you run pytest in (put the test file beside it).

Which cases. Write them down, and add one column people usually leave out — the state the test starts from:

| Case | Starting state | Action | Expected | |---|---|---|---| | Normal total | pen × 3, bag × 1 | total() | 895.0 | | Item count | pen × 3, bag × 1 | count() | 4 | | One more line | pen × 3, bag × 1 | add("notebook", 60.0, 2) | total() is 1015.0 | | Edge: nothing in it | empty cart | total() | 0 | | Invalid quantity | empty cart | add("pen", 15.0, 0) | ValueError |

Read the second column downwards. Three rows start from the same filled cart, two from an empty one. Every repeated value in that column is a fixture waiting to be written — here, empty_cart and a cart built on it. And a row that starts from a state nobody else shares does not need a fixture at all; it builds its own.

Ask one more question of each starting state: does creating it change anything outside the test? A new Cart does not. A registered discount code in a global dictionary, an opened connection, a written file — those do, and that is what tells you a fixture needs cleanup (yield, later in this chapter).

What not to test. Do not write tests proving that pytest builds a fresh fixture for each test; that is pytest's promise, and this chapter shows it once so you can trust it. Do test your own cleanup when it restores global state — that code is yours, and it can be wrong.

The rest of the chapter turns this plan into code.

A first fixture

Move the setup into a function, and put @pytest.fixture above it:

import pytest

from cart import Cart


@pytest.fixture
def cart():
    c = Cart()
    c.add("pen", 15.0, 3)
    c.add("bag", 850.0)
    return c


def test_total(cart):
    assert cart.total() == 895.0


def test_count(cart):
    assert cart.count() == 4


def test_add_one_more(cart):
    cart.add("notebook", 60.0, 2)
    assert cart.total() == 1015.0

pytest -q
...                                                                      [100%]
3 passed in 0.01s

Nothing in the tests calls cart(). Each test simply has a parameter called cart, and that is the whole request.

Before running a test, pytest looks at its parameter names. For each name it finds a fixture with that name, runs it, and passes whatever it returned in as the argument. The test says what it needs; it does not build it. This arrangement has a name — dependency injection — but the mechanism is no more than "match the parameter name to a fixture name".

The test now reads as one line of intent. And the setup lives in one place, so a change to Cart() is a change to one function.

Every test gets a fresh one

test_add_one_more adds a notebook to the cart. Did the tests that ran after it see a cart with a notebook in it? No — and that is worth seeing clearly, because it is the property that makes fixtures safe.

First, the version without a fixture. A list at the top of the module, shared by two tests:

ITEMS = ["pen", "bag"]


def test_add_item():
    ITEMS.append("notebook")
    assert len(ITEMS) == 3


def test_starts_with_two():
    assert len(ITEMS) == 2

pytest -q
.F                                                                       [100%]
=================================== FAILURES ===================================
_____________________________ test_starts_with_two _____________________________

    def test_starts_with_two():
>       assert len(ITEMS) == 2
E       AssertionError: assert 3 == 2
E        +  where 3 = len(['pen', 'bag', 'notebook'])

test_shared.py:10: AssertionError
=========================== short test summary info ============================
FAILED test_shared.py::test_starts_with_two - AssertionError: assert 3 == 2
1 failed, 1 passed in 0.01s

The second test is correct; it failed because of what the first test did. Run it on its own and it passes:

pytest -q test_shared.py::test_starts_with_two
.                                                                        [100%]
1 passed in 0.01s

A test whose result depends on which tests ran before it is one of the most expensive bugs in a test suite, because it shows up only sometimes.

Now the same two tests with the list coming from a fixture. A print inside the fixture shows when it runs; -s tells pytest not to capture printed output, so we can see it:

import pytest


@pytest.fixture
def items():
    print("building a new list")
    return ["pen", "bag"]


def test_add_item(items):
    items.append("notebook")
    assert len(items) == 3


def test_starts_with_two(items):
    assert len(items) == 2

pytest -q -s
building a new list
.building a new list
.
2 passed in 0.01s

"building a new list" appears twice. The fixture function ran once for each test that asked for it, and each test got a list nobody else had touched. That is the default, and it has a name, function scope: the fixture's value lives as long as one test function. Chapter eight shows how to change that, and why you usually should not.

Fixtures can use fixtures

A fixture can request another fixture in exactly the same way a test does — by naming it as a parameter:

import pytest

from cart import Cart


@pytest.fixture
def empty_cart():
    return Cart()


@pytest.fixture
def cart(empty_cart):
    empty_cart.add("pen", 15.0, 3)
    empty_cart.add("bag", 850.0)
    return empty_cart


def test_new_cart_is_empty(empty_cart):
    assert empty_cart.total() == 0


def test_total(cart):
    assert cart.total() == 895.0

pytest -q
..                                                                       [100%]
2 passed in 0.01s

cart builds on empty_cart, so a test can ask for whichever level of preparation it needs. Neither test calls anything; pytest follows the names, from test_total to cart and from cart to empty_cart, and builds them in the order that chain requires.

Seeing it happen — --setup-show

You do not have to add print calls to find out which fixtures ran. --setup-show prints every setup and teardown:

pytest -q --setup-show

        SETUP    F empty_cart
        test_cart.py::test_new_cart_is_empty (fixtures used: empty_cart) .
        TEARDOWN F empty_cart
        SETUP    F empty_cart
        SETUP    F cart (fixtures used: empty_cart)
        test_cart.py::test_total (fixtures used: cart, empty_cart) .
        TEARDOWN F cart
        TEARDOWN F empty_cart
2 passed in 0.01s

Read it from the top. The F means function scope — built for one test and thrown away after it. empty_cart is set up twice, once per test, which is the freshness from the last section. Where cart is used, empty_cart is set up first, because cart depends on it. And the teardowns come in the opposite order to the setups: last built, first removed.

(If you have plugins installed, extra lines for their own fixtures may appear. Those are theirs, not yours.)

So far teardown did nothing visible, because there was nothing to undo. That changes next.

yield — setup, then cleanup

Some setup leaves something behind that must be undone: a connection to close, a file to delete, a global setting to put back. For that, a fixture uses yield instead of return:

import pytest


@pytest.fixture
def connection():
    print("\nconnection: open")
    conn = {"orders": []}
    yield conn
    print("connection: close")


def test_starts_empty(connection):
    print("test: running")
    assert connection["orders"] == []

pytest -q -s

connection: open
test: running
.connection: close

1 passed in 0.01s

Everything before yield is setup. The value after yield is what the test receives. Then the fixture pauses — the test runs — and when the test is finished, pytest resumes the fixture after the yield, and that remainder is the teardown. (The . sits in front of "connection: close" because the test's result is printed the moment the test ends, just before teardown starts.)

The cleanup sits in the same function as the setup, a few lines below it. You cannot add one and forget the other without it being visible.

With two yield fixtures, one depending on the other, the order is the one --setup-show hinted at:

import pytest


@pytest.fixture
def connection():
    print("\n1. connection: open")
    conn = {"orders": []}
    yield conn
    print("5. connection: close")


@pytest.fixture
def saved_order(connection):
    print("2. saved_order: insert")
    connection["orders"].append("A-100")
    yield "A-100"
    connection["orders"].remove("A-100")
    print("4. saved_order: delete")


def test_order_is_saved(connection, saved_order):
    print("3. test: running")
    assert saved_order in connection["orders"]

pytest -q -s

1. connection: open
2. saved_order: insert
3. test: running
.4. saved_order: delete
5. connection: close

1 passed in 0.01s

Setup goes from the bottom of the dependency chain up; teardown goes back down. That is the only order that works: saved_order deletes its row through the connection, so the connection must still be open when it does.

Teardown runs even when the test fails

Why go to the trouble of a fixture for cleanup, when the test could simply clean up at its end? Try it. OPEN stands for anything a test can leave behind — an open connection, a temporary file, a changed setting:

OPEN = []


def test_broken():
    OPEN.append("connection")
    assert OPEN == ["A-100"]
    OPEN.remove("connection")


def test_nothing_left_open():
    assert OPEN == []

pytest -q --tb=no
FF                                                                       [100%]
=========================== short test summary info ============================
FAILED test_cleanup_in_body.py::test_broken - AssertionError: assert ['connec...
FAILED test_cleanup_in_body.py::test_nothing_left_open - AssertionError: asse...
2 failed in 0.01s

(--tb=no hides the tracebacks and keeps only the summary.) One mistake, two failures. A failing assert raises, and raising ends the test on the spot, so the cleanup line below it never ran. The connection stayed in OPEN, and the next test — which was perfectly correct — failed on what it inherited. In a large suite, that second failure is the one you would spend an afternoon on.

Now the same work, with the cleanup after a yield:

import pytest

OPEN = []


@pytest.fixture
def connection():
    OPEN.append("connection")
    yield "connection"
    OPEN.remove("connection")


def test_broken(connection):
    assert OPEN == ["A-100"]


def test_nothing_left_open():
    assert OPEN == []

pytest -q --tb=no
F.                                                                       [100%]
=========================== short test summary info ============================
FAILED test_cleanup_in_fixture.py::test_broken - AssertionError: assert ['con...
1 failed, 1 passed in 0.01s

One mistake, one failure. The broken test is still reported — the fixture hides nothing — but its mess did not reach the next test.

That is the reason, and the rule that follows from it: cleanup never goes at the end of a test body. It goes after the yield of a fixture.

A fixture or a plain function?

A fixture is not the only way to share setup. An ordinary function — say cart_with(*lines), which builds a cart from whatever lines it is given — works too, and the complete example below uses one.

Here the difference that decides it: a test calls a helper, and can pass it arguments. A test cannot call a fixture; it can only name it. A fixture always produces the same thing for every test that asks.

So:

• Use a fixture when many tests need the same starting state, and especially when that state needs cleaning up afterwards. Only a fixture gets a guaranteed teardown.
• Use a helper function when each test needs a slightly different object — a cart with these lines, a user with that name. Arguments are what functions are for.

They combine well: a fixture can call a helper, as the next example does. (There is also a way to make a fixture that hands back a function — the "factory" pattern in chapter twelve. You do not need it yet.)

A complete example

A shop where discount codes live in a module-level dictionary. Registering a code changes global state, so a test that registers one must remove it again.

shop.py:

DISCOUNTS = {}


def register_discount(code, percent):
    DISCOUNTS[code] = percent


def remove_discount(code):
    del DISCOUNTS[code]


class Cart:
    def __init__(self):
        self.lines = []

    def add(self, name, price, quantity=1):
        if quantity < 1:
            raise ValueError(f"quantity must be at least 1: {quantity}")
        self.lines.append((name, price, quantity))

    def total(self, code=None):
        amount = sum(price * quantity for _, price, quantity in self.lines)
        if code is not None:
            amount = amount * (100 - DISCOUNTS[code]) / 100
        return round(amount, 2)

test_shop.py:

import pytest

from shop import Cart, register_discount, remove_discount


def cart_with(*lines):
    # A plain helper: it takes arguments, so each test can ask for its own cart.
    cart = Cart()
    for name, price, quantity in lines:
        cart.add(name, price, quantity)
    return cart


@pytest.fixture
def cart():
    # The cart most tests share. Built fresh for every test that asks.
    return cart_with(("pen", 15.0, 3), ("bag", 850.0, 1))


@pytest.fixture
def summer_code():
    # Changes global state, so it must undo that change afterwards.
    register_discount("SUMMER", 10)
    yield "SUMMER"
    remove_discount("SUMMER")


def test_total(cart):
    assert cart.total() == 895.0


def test_summer_discount(cart, summer_code):
    assert cart.total(summer_code) == 805.5


def test_code_is_gone_afterwards(cart):
    with pytest.raises(KeyError):
        cart.total("SUMMER")


def test_small_cart(summer_code):
    cart = cart_with(("pen", 15.0, 2))
    assert cart.total(summer_code) == 27.0


def test_zero_quantity_is_refused():
    with pytest.raises(ValueError):
        Cart().add("pen", 15.0, 0)

pytest -q --setup-show

        SETUP    F cart
        test_shop.py::test_total (fixtures used: cart) .
        TEARDOWN F cart
        SETUP    F cart
        SETUP    F summer_code
        test_shop.py::test_summer_discount (fixtures used: cart, summer_code) .
        TEARDOWN F summer_code
        TEARDOWN F cart
        SETUP    F cart
        test_shop.py::test_code_is_gone_afterwards (fixtures used: cart) .
        TEARDOWN F cart
        SETUP    F summer_code
        test_shop.py::test_small_cart (fixtures used: summer_code) .
        TEARDOWN F summer_code
        test_shop.py::test_zero_quantity_is_refused .
5 passed in 0.01s

Three things are worth noticing.

First, each test asks only for what it uses. test_small_cart wanted a different cart, so it did not request cart at all; it called the helper with its own lines — and --setup-show confirms cart was never built for it. test_zero_quantity_is_refused needs only an empty cart and one call, so it uses no fixture; that is the "starting state nobody else shares" row from the plan.

Second, test_code_is_gone_afterwards is a test of the fixture. It runs after test_summer_discount and checks that SUMMER no longer exists. To see why it matters, change summer_code to return "SUMMER" instead of the yield and the cleanup line:

pytest -q --tb=no
..F..                                                                    [100%]
=========================== short test summary info ============================
FAILED test_shop.py::test_code_is_gone_afterwards - Failed: DID NOT RAISE Key...
1 failed, 4 passed in 0.01s

The code leaked out of the test that registered it and into the next one. Without that check, the leak would have been silent.

Third, the helper and the fixture work together. cart_with knows how to build a cart; the cart fixture decides which cart most tests share.

When it breaks

fixture 'crat' not found

_________________________ ERROR at setup of test_empty _________________________
file .../test_typo.py, line 11
  def test_empty(crat):
E       fixture 'crat' not found
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, capteesys, cart, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, subtests, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

A parameter name matched no fixture. Either it is misspelled — here crat for cart, which is right there in the list — or the function is missing its @pytest.fixture line, in which case it will be absent from the list. Note the E and ERROR, not F and FAILED: the test never started.

Fixture "cart" called directly

Fixture "cart" called directly. Fixtures are not meant to be called directly,
but are created automatically when test functions request them as parameters.

The test wrote cart() in its body. A fixture is requested by naming it as a parameter, never by calling it. If you really do want to call it with your own arguments, what you want is a helper function.

ERROR at setup of test_total instead of a failure The exception was raised inside the fixture, before the test ran — for example ValueError: quantity must be at least 1: 0 from a bad add() in the setup. An E means "the preparation broke"; an F means "the claim was false". Look in the fixture first.

fixture function has more than one 'yield' A fixture yields exactly once. Setup goes above the yield, cleanup goes below it; for two values, yield a tuple or write two fixtures.

A test passes alone and fails among the others Something is shared between tests — a module-level list or dictionary, or a global that a fixture changed and did not put back. Build it in a fixture, and if it is global state, undo the change after yield.

The cleanup did not run The cleanup is in the test body after a failing assert, or the fixture uses return. Move it into a yield fixture, after the yield.

Your turn

Write library.py with a module-level LOG = [] and a Library class: add(title); borrow(title), which raises ValueError if the title is not on the shelf and otherwise removes it and appends "borrow <title>" to LOG; give_back(title); and available(), returning a sorted list of the titles on the shelf.

Then write test_library.py. Plan it first — a table with a starting state column, as at the start of this chapter — and then write:

1. A fixture library returning a Library with three books, and at least three tests that use it — one of which borrows a book, so that a later test can show the shelf is full again
2. A fixture borrowed that depends on library, borrows one book and returns its title, with two tests that use it
3. A yield fixture clean_log that empties LOG before the test and again after it; one test that uses it, and one test after that which asserts LOG == []
4. A plain helper library_with(*titles), used by library and by one test that needs an empty shelf

Run pytest -q, then pytest -q --setup-show, and check every line against what you expected.

Finally, break it on purpose: in clean_log, replace the yield and the line after it with return LOG. Which test fails, and is it the one that used the fixture?

Solution

The plan first:

| Case | Starting state | Action | Expected | |---|---|---|---| | Three books on the shelf | Dune, Emma, Ulysses | available() | all three, sorted | | Borrowing removes a book | same | borrow("Dune") | Emma, Ulysses | | No leak between tests | same | — | still three | | Cannot borrow twice | same, Emma borrowed | borrow("Emma") | ValueError | | Giving back | same, Emma borrowed | give_back("Emma") | Emma on the shelf | | Borrowing is logged | same, LOG empty | borrow("Ulysses") | LOG == ["borrow Ulysses"] | | Log cleaned up | after the test above | — | LOG == [] | | Edge: empty shelf | no books | available() | [] |

Two starting states repeat — the three-book shelf and the shelf with Emma out — so they became fixtures. The empty log is needed only once, but it touches global state that must be put back afterwards, and only a fixture guarantees that; so it is a fixture too. The empty shelf is needed once and leaves nothing behind, so it uses the helper.

library.py:

LOG = []


class Library:
    def __init__(self):
        self.shelf = set()

    def add(self, title):
        self.shelf.add(title)

    def borrow(self, title):
        if title not in self.shelf:
            raise ValueError(f"not on the shelf: {title}")
        self.shelf.remove(title)
        LOG.append(f"borrow {title}")

    def give_back(self, title):
        self.shelf.add(title)

    def available(self):
        return sorted(self.shelf)

test_library.py:

import pytest

from library import LOG, Library


def library_with(*titles):
    # Helper: each caller chooses its own shelf.
    library = Library()
    for title in titles:
        library.add(title)
    return library


@pytest.fixture
def library():
    return library_with("Dune", "Emma", "Ulysses")


@pytest.fixture
def borrowed(library):
    library.borrow("Emma")
    return "Emma"


@pytest.fixture
def clean_log():
    LOG.clear()
    yield LOG
    LOG.clear()


def test_three_books_on_the_shelf(library):
    assert library.available() == ["Dune", "Emma", "Ulysses"]


def test_borrow_removes_the_book(library):
    library.borrow("Dune")
    assert library.available() == ["Emma", "Ulysses"]


def test_next_test_still_sees_three(library):
    assert len(library.available()) == 3


def test_cannot_borrow_twice(library, borrowed):
    with pytest.raises(ValueError):
        library.borrow(borrowed)


def test_give_back_returns_it(library, borrowed):
    library.give_back(borrowed)
    assert borrowed in library.available()


def test_borrow_is_logged(library, clean_log):
    library.borrow("Ulysses")
    assert clean_log == ["borrow Ulysses"]


def test_log_is_empty_afterwards():
    assert LOG == []


def test_empty_library():
    library = library_with()
    assert library.available() == []

pytest -q
........                                                                 [100%]
8 passed in 0.01s

With --setup-show you will see SETUP F library once for each of the six tests that use it, borrowed set up after library and torn down before it, and no fixture lines at all for the last two tests.

And the experiment, with return LOG in place of the yield:

pytest -q --tb=no
......F.                                                                 [100%]
=========================== short test summary info ============================
FAILED test_library.py::test_log_is_empty_afterwards - AssertionError: assert...
1 failed, 7 passed in 0.01s

The decisions, one at a time:

• library returns, it does not yield. A new Library changes nothing outside the test, so there is nothing to undo. yield is for fixtures with something to clean up, not a style choice.
• test_next_test_still_sees_three is there on purpose. It runs right after a test that borrowed Dune, and passes because library was built again. In --setup-show you can count it: SETUP F library once per test.
• borrowed returns the title. The tests use borrowed instead of repeating "Emma", so if the fixture changes which book it borrows, the tests follow. And because borrowed depends on library, both tests that ask for both receive the same library — the one Emma was taken from.
• clean_log clears on both sides of the yield. Before, because other tests that call borrow() — test_borrow_removes_the_book, the borrowed fixture — have already written to LOG; the test must start from a known state. After, so nothing leaks to the next test.
• *The experiment fails in the other test.* test_borrow_is_logged still passes; test_log_is_empty_afterwards, which did nothing wrong, fails. That is what a missing cleanup always looks like: the failure shows up somewhere else. And it is exactly why the solution has a test that checks the cleanup.
• library_with is a helper, not a fixture, because the one test that uses it wants a different shelf, and only a function can take arguments. library calls it too, so the knowledge of how a library is filled lives in one place.