pythontojob
Log inSign up
Chapter 12

Markers, skip and xfail — which tests run where

Tell pytest what each test needs and what you expect of it with skip, skipif, importorskip and xfail; what strict=True, raises= and XPASS mean; register your own markers and select with -m; plus parametrized and factory fixtures.

50 minPython 3.12
  1. 1Encounter
  2. 2Understand
  3. 3Worked
  4. 4Predict
  5. 5Apply
  6. 6Stretch

The problem we are solving

A small project, shop, with one module and two test files. money.py:

python
def parse_price(text: str) -> float:
    return float(text.replace(",", ""))


def split_bill(total: float, people: int) -> float:
    return round(total / people, 2)

test_money.py checks it with plain values. test_export.py reads prices from YAML, which needs the third-party yaml package:

python
import yaml

from money import parse_price


def test_prices_from_yaml():
    data = yaml.safe_load("pen: '15.00'")
    assert parse_price(data["pen"]) == 15.0

On a machine where yaml is not installed, pytest -q says:

text
==================================== ERRORS ====================================
_______________________ ERROR collecting test_export.py ________________________
ImportError while importing test module '/home/you/shop/test_export.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
test_export.py:1: in <module>
    import yaml
E   ModuleNotFoundError: No module named 'yaml'
=========================== short test summary info ============================
ERROR test_export.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
1 error in 0.13s

Read the last two lines. Interrupted. The two good tests in test_money.py never ran. One file that could not run on this machine took the whole run down with it.

That is the real situation in any project that lives long enough. Some tests need a library that is optional. Some only make sense on Linux. Some need an API key that only CI has. Some describe a bug everyone knows about and nobody has fixed yet. Some take a minute, and you do not want to wait a minute every time you save a file.

Deleting those tests is wrong, and so is commenting them out. What you want is to tell pytest what each test needs and what you expect of it, and let pytest decide, run by run, what to do with it. The tool for that is the marker.

By the end of this chapter you can

  • Skip a test always, on a condition, or from inside the test, and make the reason visible
  • Skip a whole file when an optional package is missing, with pytest.importorskip
  • Mark a known bug with xfail, and make it precise with strict=True and raises=
  • Read the report letters s, x and X, and ask for details with -rs, -rx, -rA
  • Invent your own markers, register them, and catch typos with --strict-markers
  • Select tests with -m "not slow" and -m "slow and db"
  • Write a fixture that runs every test several times, and a fixture that hands back a function

Prerequisites: mocking.

Before you write the test

Markers are not about what a test checks — earlier chapters covered that. They are about two questions you answer before the test is written, and that most people only answer after a run has gone red for the wrong reason.

1. What does this test need in order to run at all? A package that is not a hard dependency, an operating system, a Python version, an environment variable holding a secret, a database, a network, a lot of time. Each of these is a precondition. If it is missing, the test has nothing to say: it did not pass, and it did not fail.

2. What do I expect it to do today? Normally, pass. But if the code has a known bug and you are writing the test that describes the correct behaviour, you expect it to fail — and to fail in one particular way.

The answers decide the marker, so write them down first. For the shop project the plan looks like this:

| Case | Input / situation | Needs | Expected | Marker | |---|---|---|---|---| | plain price | "1,250.50" | nothing | 1250.5 | none | | YAML export | "pen: '15.00'" | the yaml package | 15.0 where installed | pytest.importorskip("yaml") | | POSIX path | "data", "prices.csv" | not Windows | "data/prices.csv" | skipif(sys.platform == "win32") | | live rates | the real API | RATES_API_KEY set | a rate, only where the key exists | pytest.skip() inside the test | | currency symbol | "$1,250" | nothing | 1250.0 — today ValueError (bug #42) | xfail(raises=ValueError, strict=True) | | many prices | 1000 strings | a second or more | correct sum | slow (custom) |

Read the table by column. The Needs column produces skips. The Expected column produces xfails. The time it takes produces a custom marker that lets you choose what to run.

What you must have in place before any of this works:

  • a virtual environment with pytest installed, and money.py importable from the tests (run from the project root, as in earlier chapters);
  • a pyproject.toml at the project root, because that is where custom markers are registered;
  • the bug written down somewhere with a number, so the reason= can point at it.

And what not to mark:

  • Do not skip a test because it fails and you do not know why. A skip hides the failure from every later run. Either fix it, or understand it well enough to write xfail(raises=...) with a reason.
  • Do not mark a flaky test xfail. A test that sometimes passes will show as X some days and x others, and with strict=True it will fail at random. Flakiness is a bug in the test; fix that instead.
  • Do not label everything slow. If most of the suite is deselected on every run, the everyday run checks nothing. Label the few tests that actually cost seconds.
  • Do not test that pytest skips correctly. You are testing money.py, not pytest. The marker is configuration; the run report is enough proof it works.

The rest of the chapter implements this table, row by row.

skip — not this test, and here is why

First the way it is usually done before anyone knows about markers — an early return:

python
import os


def test_live_rates():
    if "RATES_API_KEY" not in os.environ:
        return
    assert False, "would call the real API here"
text
.                                                                        [100%]
1 passed in 0.07s

Passed. The test checked nothing at all, and the report says it succeeded. On a laptop without the key that is a lie told every single run, and the green dot is indistinguishable from a real one. The fix is not to avoid the condition but to report it honestly — as a skip.

The simplest marker is a decorator that says "do not run this", with a reason attached:

python
import os
import sys

import pytest

from money import parse_price, split_bill


@pytest.mark.skip(reason="rounding rules not agreed yet")
def test_split_rounding():
    assert split_bill(100.0, 3) == 33.34


@pytest.mark.skipif(sys.platform == "win32", reason="uses a POSIX path")
def test_posix_path():
    assert os.path.join("data", "prices.csv") == "data/prices.csv"


@pytest.mark.skipif(sys.version_info < (3, 13), reason="needs Python 3.13")
def test_new_feature():
    assert parse_price("10") == 10.0


def test_live_rates():
    if "RATES_API_KEY" not in os.environ:
        pytest.skip("RATES_API_KEY is not set")
    assert False, "would call the real API here"
text
s.ss                                                                     [100%]
1 passed, 3 skipped in 0.09s

Four tests, four different ways of deciding:

  • @pytest.mark.skip(reason=...) — always skipped. The body never runs.
  • @pytest.mark.skipif(condition, reason=...) — skipped when the condition is true. The condition is evaluated when the file is collected. This run was on Linux, so test_posix_path ran and passed; the Python was 3.12, so test_new_feature was skipped.
  • pytest.skip("...") called inside the test — the decision is made while the test is running. Use it when you only find out halfway, for example after looking at the environment or at what a fixture returned.

Each s in the progress line is a skipped test. A skip is not a failure: the run is green.

But the reasons are nowhere to be seen. By default pytest summarises failures and errors only. Ask for skips with -rs:

text
s.ss                                                                     [100%]
=========================== short test summary info ============================
SKIPPED [1] test_skips.py:9: rounding rules not agreed yet
SKIPPED [1] test_skips.py:19: needs Python 3.13
SKIPPED [1] test_skips.py:26: RATES_API_KEY is not set
1 passed, 3 skipped in 0.10s

That is why the reason matters. Three months from now, "skipped" alone tells nobody whether the test can be switched back on.

importorskip — when a whole file needs a package

Back to test_export.py. Replace import yaml with this:

python
import pytest

from money import parse_price

yaml = pytest.importorskip("yaml")


def test_prices_from_yaml():
    data = yaml.safe_load("pen: '15.00'")
    assert parse_price(data["pen"]) == 15.0
text
..                                                                       [100%]
=========================== short test summary info ============================
SKIPPED [1] test_export.py:5: could not import 'yaml': No module named 'yaml'
2 passed, 1 skipped in 0.08s

pytest.importorskip("yaml") tries the import. If it works, it returns the module, which is why it is assigned to yaml. If it fails, the whole file is skipped, with the import error as the reason. The other two tests now run. Install yaml and the same file runs in full, with no change.

xfail — a bug you know about

A skip says "this test cannot run here". There is a second, different statement: "this test runs, and it is expected to fail, because the code has a known bug".

parse_price has one. It does not handle a currency symbol:

python
import pytest

from money import parse_price


@pytest.mark.xfail(reason="bug #42: currency symbol is not stripped")
def test_parse_with_symbol():
    assert parse_price("$1,250") == 1250.0


@pytest.mark.xfail(reason="spaces as thousands separators")
def test_parse_with_spaces():
    assert parse_price("1 250") == 1250.0

pytest -q -rx:

text
xx                                                                       [100%]
=========================== short test summary info ============================
XFAIL test_xfail.py::test_parse_with_symbol - bug #42: currency symbol is not stripped
XFAIL test_xfail.py::test_parse_with_spaces - spaces as thousands separators
2 xfailed in 0.08s

Unlike a skipped test, an xfail test does run. It failed, as predicted, so it shows as a lowercase x and the run stays green. The test is now a written record of the bug, sitting in the suite, rather than a note in someone's head.

Why not simply skip it? Because a skipped test never runs, so it can never tell you anything — not when the bug is fixed, and not when the code changes so that it fails in a new way. An xfail test runs every time, and its result is compared with what you predicted. Skip is for "cannot run here"; xfail is for "runs, and I know what it will say".

Now someone fixes bug #42 and changes money.py to strip the $:

text
Xx                                                                       [100%]
=================================== XPASSES ====================================
=========================== short test summary info ============================
XPASS test_xfail.py::test_parse_with_symbol - bug #42: currency symbol is not stripped
1 xfailed, 1 xpassed in 0.12s

A capital X, XPASS: "expected to fail, but passed". And the run is still green — the exit code is 0. Nobody looks at a green run, so the marker stays, and the test goes on claiming a bug that no longer exists.

strict=True — make good news loud

python
import pytest

from money import parse_price


@pytest.mark.xfail(reason="bug #42: currency symbol is not stripped", strict=True)
def test_parse_with_symbol():
    assert parse_price("$1,250") == 1250.0
text
F                                                                        [100%]
=================================== FAILURES ===================================
____________________________ test_parse_with_symbol ____________________________
[XPASS(strict)] bug #42: currency symbol is not stripped
=========================== short test summary info ============================
FAILED test_xfail.py::test_parse_with_symbol - [XPASS(strict)] bug #42: curre...
1 failed in 0.10s

With strict=True, an unexpected pass is a failure. That sounds backwards, and it is exactly what you want: the person who fixed the bug is told to remove the marker, and the test turns into an ordinary test that guards the fix from then on. If you want this for every xfail in the project, set xfail_strict = true in the pytest configuration.

raises= — fail for the right reason

An xfail without arguments accepts any failure. That includes the ones you did not mean:

python
import pytest

from money import parse_price


@pytest.mark.xfail(reason="bug #42")
def test_loose():
    assert parse_prise("$1,250") == 1250.0


@pytest.mark.xfail(raises=ValueError, reason="bug #42")
def test_precise():
    assert parse_prise("$1,250") == 1250.0
text
xF                                                                       [100%]
=================================== FAILURES ===================================
_________________________________ test_precise _________________________________

    @pytest.mark.xfail(raises=ValueError, reason="bug #42")
    def test_precise():
>       assert parse_prise("$1,250") == 1250.0
               ^^^^^^^^^^^
E       NameError: name 'parse_prise' is not defined

test_raises.py:13: NameError
=========================== short test summary info ============================
XFAIL test_raises.py::test_loose - bug #42
1 failed, 1 xfailed in 0.09s

Both tests have the same typo, parse_prise. test_loose swallowed the NameError and reported a calm x — it would have stayed "expected to fail" forever, without ever testing parse_price. test_precise declared which failure it expects, the ValueError that float("$1250") raises, so any other exception is reported as a real failure. Correct the typo and both become x.

The report letters

The progress line uses one character per test, and -r takes the same letters to choose what goes into the summary:

| Letter | Meaning | Shown in the summary with | |---|---|---| | . | passed | -rp | | F | failed | -rf (on by default) | | E | error in setup or teardown | -rE (on by default) | | s | skipped | -rs | | x | xfailed — failed as expected | -rx | | X | xpassed — passed unexpectedly | -rX |

Letters combine: -rsx shows skips and xfails together. -rA shows all of them, passes included:

text
..xs                                                                     [100%]
==================================== PASSES ====================================
=========================== short test summary info ============================
PASSED test_params.py::test_parse_price[15-15.0]
PASSED test_params.py::test_parse_price[1,250.50-1250.5]
SKIPPED [1] test_params.py:6: space separators not decided
XFAIL test_params.py::test_parse_price[$1,250-1250.0] - bug #42
2 passed, 1 skipped, 1 xfailed in 0.08s

Your own markers

skip and xfail change what pytest does with a test. A marker can also be a plain label, with a name you invent, that changes nothing until you use it to choose tests. Say some tests are slow, and some need a database:

python
import time

import pytest

from money import parse_price, split_bill


def test_parse_plain():
    assert parse_price("1,250.50") == 1250.5


@pytest.mark.slow
def test_parse_many():
    time.sleep(1)
    assert sum(parse_price("1.5") for _ in range(1000)) == 1500.0


@pytest.mark.slow
@pytest.mark.db
def test_bill_from_database():
    time.sleep(1)
    assert split_bill(300.0, 3) == 100.0


@pytest.mark.db
def test_bill_lookup():
    assert split_bill(90.0, 3) == 30.0
text
....                                                                     [100%]
=============================== warnings summary ===============================
test_marks.py:12
  /home/you/shop/test_marks.py:12: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.slow

test_marks.py:18
  /home/you/shop/test_marks.py:18: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.slow

test_marks.py:19
  /home/you/shop/test_marks.py:19: PytestUnknownMarkWarning: Unknown pytest.mark.db - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.db

test_marks.py:25
  /home/you/shop/test_marks.py:25: PytestUnknownMarkWarning: Unknown pytest.mark.db - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.db

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
4 passed, 4 warnings in 2.09s

All four passed, and pytest complains about every use. pytest.mark.<anything> is accepted — the attribute is created on the spot — so pytest cannot tell slow from a misspelling of something else. It asks you to declare the names you mean.

Registering markers

The list goes in pyproject.toml, one string per marker, the name before the colon and a description after it:

toml
[tool.pytest.ini_options]
markers = [
    "slow: takes more than a second; deselect with -m 'not slow'",
    "db: needs the database",
]
text
....                                                                     [100%]
4 passed in 2.11s

The warnings are gone, and pytest --markers now lists them for the next person to join the project:

text
@pytest.mark.slow: takes more than a second; deselect with -m 'not slow'

@pytest.mark.db: needs the database

Selecting with -m

Now the labels pay for themselves. -m takes an expression over marker names, with and, or, not and parentheses:

text
$ pytest -q -m "not slow"
..                                                                       [100%]
2 passed, 2 deselected in 0.07s

$ pytest -q -m "slow and db"
.                                                                        [100%]
1 passed, 3 deselected in 1.08s

$ pytest -q -m "slow or db"
...                                                                      [100%]
3 passed, 1 deselected in 2.07s

Deselected is not skipped. A skipped test was chosen and then declined to run; a deselected test was never chosen. -m "not slow" is the run you do on every save; the full run is for CI and before you push.

--strict-markers — a typo becomes an error

The warning is easy to miss. Here is a test someone added later, with the marker misspelled:

python
@pytest.mark.slwo
def test_split_large_group():
    time.sleep(1)
    assert split_bill(1000.0, 8) == 125.0
text
...                                                                      [100%]
=============================== warnings summary ===============================
test_marks.py:30
  /home/you/shop/test_marks.py:30: PytestUnknownMarkWarning: Unknown pytest.mark.slwo - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.slwo

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
3 passed, 2 deselected, 1 warning in 1.07s

-m "not slow" was asked for, and the run still took a second: the misspelled test is not slow, so it ran. With --strict-markers, an unregistered marker stops collection instead:

text
==================================== ERRORS ====================================
________________________ ERROR collecting test_marks.py ________________________
'slwo' not found in `markers` configuration option
=========================== short test summary info ============================
ERROR test_marks.py - Failed: 'slwo' not found in `markers` configuration option
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
1 error in 0.13s

You do not want to type that flag every time, so put it in addopts:

toml
[tool.pytest.ini_options]
addopts = "--strict-markers"
markers = [
    "slow: takes more than a second; deselect with -m 'not slow'",
    "db: needs the database",
]

pytestmark — mark a whole file

When every test in a file needs the database, decorating each one is repetition. A module-level variable named pytestmark applies to every test in the file:

python
import pytest

from money import split_bill

pytestmark = pytest.mark.db


def test_bill_for_two():
    assert split_bill(50.0, 2) == 25.0


def test_bill_for_four():
    assert split_bill(80.0, 4) == 20.0
text
$ pytest -q -m db
..                                                                       [100%]
2 passed, 4 deselected in 0.07s

The name must be exactly pytestmark. It can also be a list, pytestmark = [pytest.mark.db, pytest.mark.slow].

Marks on a single case of parametrize

A marker on a parametrized test applies to every case. To mark one case, wrap it in pytest.param(..., marks=...):

python
import pytest

from money import parse_price


@pytest.mark.parametrize(
    "text, expected",
    [
        ("15", 15.0),
        ("1,250.50", 1250.5),
        pytest.param(
            "$1,250", 1250.0,
            marks=pytest.mark.xfail(raises=ValueError, reason="bug #42"),
        ),
        pytest.param(
            "1 250", 1250.0,
            marks=pytest.mark.skip(reason="space separators not decided"),
        ),
    ],
)
def test_parse_price(text, expected):
    assert parse_price(text) == expected
text
test_params.py::test_parse_price[15-15.0] PASSED                         [ 25%]
test_params.py::test_parse_price[1,250.50-1250.5] PASSED                 [ 50%]
test_params.py::test_parse_price[$1,250-1250.0] XFAIL (bug #42)          [ 75%]
test_params.py::test_parse_price[1 250-1250.0] SKIPPED (space separa...) [100%]
=================== 2 passed, 1 skipped, 1 xfailed in 0.11s ====================

The known bug lives next to the cases that work, in the same table, and the day it is fixed you delete one pytest.param wrapper.

Going further: fixtures that vary, and fixtures that build

Two fixture patterns did not fit into the fixture chapters, and markers make a good moment for them, because both multiply or shape the tests you just learned to select.

A parametrized fixture

parametrize varies the inputs of one test. Sometimes you want every test that uses a fixture to run once per value of that fixture. Give @pytest.fixture a params list, and read the current value from request.param:

python
import pytest

from money import parse_price


@pytest.fixture(params=["1250", "1,250", "1,250.00"], ids=["plain", "comma", "decimals"])
def price_text(request):
    return request.param


def test_parses_to_1250(price_text):
    assert parse_price(price_text) == 1250.0


def test_is_positive(price_text):
    assert parse_price(price_text) > 0
text
test_fixtures.py::test_parses_to_1250[plain] PASSED                      [ 16%]
test_fixtures.py::test_parses_to_1250[comma] PASSED                      [ 33%]
test_fixtures.py::test_parses_to_1250[decimals] PASSED                   [ 50%]
test_fixtures.py::test_is_positive[plain] PASSED                         [ 66%]
test_fixtures.py::test_is_positive[comma] PASSED                         [ 83%]
test_fixtures.py::test_is_positive[decimals] PASSED                      [100%]
============================== 6 passed in 0.08s ===============================

Two tests, three values, six runs. request is a built-in fixture that describes the test being set up; request.param exists only when the fixture has params. ids names each value in the report — without it, pytest builds an id from the value itself, which for long or unprintable values is hard to read. The list may hold pytest.param(...) entries with marks= too, exactly as in parametrize.

Where this earns its place: the same set of tests against several backends, several file formats, several currencies. Add one value to params, and every test that uses the fixture covers it.

A factory fixture

A fixture cannot take arguments from the test; the test only names it. So what do you do when a test needs two objects, built differently? Return a function:

python
import pytest

from money import parse_price


@pytest.fixture
def make_receipt(tmp_path):
    def _make(name, lines):
        path = tmp_path / f"{name}.txt"
        path.write_text("\n".join(lines))
        return path

    return _make


def receipt_total(path):
    return sum(parse_price(line) for line in path.read_text().splitlines())


def test_two_receipts(make_receipt):
    lunch = make_receipt("lunch", ["15", "1,250.50"])
    taxi = make_receipt("taxi", ["300"])
    assert receipt_total(lunch) + receipt_total(taxi) == 1565.5


def test_empty_receipt(make_receipt):
    empty = make_receipt("empty", [])
    assert receipt_total(empty) == 0
text
..                                                                       [100%]
2 passed in 0.08s

The fixture still runs once per test, and still gets its own fixtures (tmp_path here, which also takes care of deleting the files). What it hands over is _make, and the test calls it as often as it likes, with whatever arguments it likes. The name make_... is the usual convention, and it tells the reader at a glance that this fixture is a function to call, not a ready-made value.

A complete example

Everything together in one project. pyproject.toml:

toml
[tool.pytest.ini_options]
addopts = "--strict-markers"
markers = [
    "slow: takes more than a second; deselect with -m 'not slow'",
    "network: talks to an outside service",
]

test_checkout.py, using the same money.py — still with bug #42:

python
import os
import time

import pytest

from money import parse_price, split_bill


# Every test that asks for amount_text runs once per format.
@pytest.fixture(params=["300", "300.00", "1,250"], ids=["whole", "decimals", "comma"])
def amount_text(request):
    return request.param


# A factory: the fixture hands back a function the test can call many times.
@pytest.fixture
def make_share():
    def _make(text, people):
        return split_bill(parse_price(text), people)

    return _make


def test_share_is_positive(amount_text, make_share):
    assert make_share(amount_text, 3) > 0


@pytest.mark.parametrize(
    "text, people, share",
    [
        ("300", 3, 100.0),
        ("90", 4, 22.5),
        pytest.param(
            "$300", 3, 100.0,
            marks=pytest.mark.xfail(raises=ValueError, strict=True, reason="bug #42"),
        ),
    ],
)
def test_share(make_share, text, people, share):
    assert make_share(text, people) == share


@pytest.mark.slow
def test_many_group_sizes(make_share):
    time.sleep(1)  # stands in for real, slow work
    assert all(make_share("300", n) > 0 for n in range(1, 1000))


@pytest.mark.network
def test_live_exchange_rate():
    if "RATES_API_KEY" not in os.environ:
        pytest.skip("RATES_API_KEY is not set")
    raise AssertionError("would call the real service here")

The everyday run, with reasons:

text
$ pytest -q -rsx -m "not slow"
.....xs                                                                  [100%]
=========================== short test summary info ============================
SKIPPED [1] test_checkout.py:52: RATES_API_KEY is not set
XFAIL test_checkout.py::test_share[$300-3-100.0] - bug #42
5 passed, 1 skipped, 1 deselected, 1 xfailed in 0.09s

Only the slow one, and the offline run that leaves out both the slow and the network tests:

text
$ pytest -q -m slow
.                                                                        [100%]
1 passed, 7 deselected in 1.07s

$ pytest -q -m "not slow and not network"
.....x                                                                   [100%]
5 passed, 2 deselected, 1 xfailed in 0.08s

Count them: one fixture with three values gives three runs of test_share_is_positive, three cases of test_share, one slow test and one network test — eight in all, which is what 1 passed, 7 deselected adds up to.

And every result tells the truth. The network test does not pretend to pass without a key; it says it was skipped and why. Bug #42 is in the suite, and because it is strict with raises=ValueError, the day it is fixed the run turns red and asks for the marker to be removed.

When it breaks

PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo? The marker is not registered. Add it to markers in pyproject.toml. If the name really is a typo, fix it — and add --strict-markers to addopts so the next one is an error rather than a warning.

`'slwo' not found in markers configuration option` This is --strict-markers doing its job: the file uses a marker that is not in the list. Fix the spelling, or register the new name.

8 deselected and no tests ran, exit code 5 -m matched nothing. A misspelled name in a -m expression is not an error, even with --strict-markers — -m sloww simply selects no test. Check the name against pytest --markers.

ERROR: Wrong expression passed to '-m': not slow and: at column 13: expected not OR left parenthesis OR identifier; got end of input The -m expression is incomplete. Quote the whole expression, and make sure every and/or has something on both sides.

Error evaluating 'skipif': you need to specify reason=STRING when using booleans as conditions. skipif(sys.platform == "linux") with no reason=. A boolean condition says nothing about why, so pytest insists on a reason.

`Using pytest.skip outside of a test will skip the entire module. If that's your intention, pass allow_module_level=True.` pytest.skip("...") was called at the top level of a file. For a missing package use pytest.importorskip; otherwise write pytest.skip("...", allow_module_level=True), or use a skipif in pytestmark.

[XPASS(strict)] bug #42 Good news, reported as a failure: the bug is fixed. Remove the xfail marker and keep the test.

An xfail test that stays x even after the bug is fixed It is failing for another reason — often a typo in the test itself. Add raises= with the exception you actually expect, and anything else will show as a real failure.

AttributeError: 'SubRequest' object has no attribute 'param' The fixture reads request.param, but it has no params= list, and was not parametrized in any other way. Add params=[...] to @pytest.fixture.

fixture 'name' not found, pointing at a fixture A fixture was written as def make_user(name):, hoping the test would pass name. Fixture arguments are other fixtures, so pytest went looking for one called name. Turn it into a factory: an inner function that takes name, returned by the fixture.