pythontojob
Log inSign up
Chapter 10

monkeypatch — controlling the environment, time and the network

Testing code that depends on environment variables, the clock, randomness or the network. setenv, delenv, setattr, setitem, chdir, the automatic undo after every test, the "patch where it is looked up" rule, and dependency injection that needs no patching at all.

50 minPython 3.12
  1. 1Encounter
  2. 2Understand
  3. 3Worked
  4. 4Predict
  5. 5Apply
  6. 6Stretch

The problem we are solving

Here is a function that decides whether an application runs in debug mode. It reads an environment variable:

settings.py:

python
import os


def debug_enabled():
    return os.environ.get("APP_DEBUG", "0") == "1"

test_settings.py:

python
from settings import debug_enabled


def test_debug_is_off_by_default():
    assert debug_enabled() is False

Run pytest -q in your terminal and it passes. Run it again on a machine where someone has exported APP_DEBUG=1 — a colleague's laptop, or a CI server set up for debugging — and the same test, on the same code, fails:

text
$ pytest -q
.                                                                        [100%]
1 passed in 0.01s

$ APP_DEBUG=1 pytest -q
F                                                                        [100%]
=================================== FAILURES ===================================
_________________________ test_debug_is_off_by_default _________________________

    def test_debug_is_off_by_default():
>       assert debug_enabled() is False
E       assert True is False
E        +  where True = debug_enabled()

test_settings.py:5: AssertionError
=========================== short test summary info ============================
FAILED test_settings.py::test_debug_is_off_by_default - assert True is False
1 failed in 0.01s

Nothing in the code changed. What changed was the world around it. Environment variables, the current time, random numbers, the current directory, the network — code that reads any of these gives a different answer depending on where and when it runs, and a test that does not control them is not testing your code alone.

The test needs to say: for the length of this test, the world looks like this — and then put everything back. That is what the built-in monkeypatch fixture does.

By the end of this chapter you can

  • Set and remove environment variables for one test with monkeypatch.setenv and monkeypatch.delenv
  • Replace a function with monkeypatch.setattr, using either an object and a name or a "module.attr" string
  • Change one key of a dictionary with monkeypatch.setitem, and the working directory with monkeypatch.chdir
  • Show that every change is undone when the test ends
  • Patch a name where it is looked up, and explain why patching random.randint misses from random import randint
  • Pass a clock or a random function in as an argument, so that no patching is needed at all

Prerequisites: built-in fixtures.

Before you write the test

With code like this, most of the work happens before the first def test_. Four questions, in order.

1. What does the function promise — and what does it read that you did not pass in? Read the function and mark every line that takes something from outside its arguments: os.environ, datetime.now(), random, a relative file path, a module-level dictionary, a network call. Each one is a hidden input. The promise of the function is "given these hidden inputs, I return this", and a test has to fix every hidden input, or it is testing whatever the machine happens to hold.

2. What must be in place? The usual: a virtual environment with pytest installed, and the module importable from the test (the test file beside it, or the project installed). Then, for this topic, one decision per hidden input:

  • an environment variable — setenv it, or delenv it with raising=False;
  • a function that reaches the network, the disk or a random source — replace it with setattr, in the module that calls it;
  • a module-level dictionary — change one key with setitem;
  • a relative path — chdir into a tmp_path;
  • a clock or random choice in code you are writing now — pass it in as an argument instead.

3. Which cases? Take the configuration loader you will build at the end of this chapter. It reads defaults, then an optional app.cfg, then APP_* environment variables. Its plan, before any code:

| Case | The world during the test | Expected | |---|---|---| | Nothing set | empty folder, no APP_* variables | port 8000, debug False, region "bd" | | File overrides defaults | app.cfg contains port = 9000 | port 9000 | | Environment overrides file | app.cfg says 9000, APP_PORT=7000 | port 7000 | | A default changes | DEFAULTS["region"] is "ae" | region "ae" | | Invalid value | APP_PORT=70000 | ValueError mentioning "out of range" | | Reader swapped out | read_file returns {"port": "1234"} | port 1234 | | Noise in your shell | APP_PORT=5 exported before running pytest | every row above still holds |

The last row is the one people forget. It is not a test of its own — it is a property of the whole suite, and it is what decides that every test must clear the variables first, not only the ones that set them.

4. What not to test. Do not test that os.environ stores strings or that urllib can fetch a page — that is Python's job, and it is tested already. Do not let a unit test touch the real network: it is slow, it fails for reasons unrelated to your code, and the weather changes. And never assert on the fake itself. This test passes and proves nothing:

python
import weather


def test_pointless(monkeypatch):
    monkeypatch.setattr(weather, "fetch_temperature", lambda city: 35)
    assert weather.fetch_temperature("Dhaka") == 35
text
.                                                                        [100%]
1 passed in 0.01s

It checks that the patch is in place, not that any of your code works. The fake exists so that other code — advice, below — can be tested on top of it. If the assertion names only the thing you replaced, delete the test.

The sections that follow implement this plan one tool at a time.

setenv and delenv — the environment, per test

monkeypatch is a fixture like tmp_path: ask for it by name in the test's parameters. Here are four tests for a slightly larger settings.py:

python
import os


def debug_enabled():
    return os.environ.get("APP_DEBUG", "0") == "1"


def api_port():
    return int(os.environ.get("APP_PORT", "8000"))
python
from settings import api_port, debug_enabled


def test_debug_on(monkeypatch):
    monkeypatch.setenv("APP_DEBUG", "1")
    assert debug_enabled() is True


def test_debug_off_by_default(monkeypatch):
    monkeypatch.delenv("APP_DEBUG", raising=False)
    assert debug_enabled() is False


def test_port_from_env(monkeypatch):
    monkeypatch.setenv("APP_PORT", "9000")
    assert api_port() == 9000


def test_port_default(monkeypatch):
    monkeypatch.delenv("APP_PORT", raising=False)
    assert api_port() == 8000
text
$ pytest -q
....                                                                     [100%]
4 passed in 0.01s

$ APP_DEBUG=1 APP_PORT=1234 pytest -q
....                                                                     [100%]
4 passed in 0.01s

The second run is the one that matters. The shell exported values that would have broken two of these tests, and all four still pass, because each test states the environment it needs.

setenv(name, value) sets a variable. delenv(name) removes one — and by default it raises KeyError if the variable was not there to begin with. A test for the "not set" case does not care whether it was set, only that it is not set now, so raising=False is what you almost always want.

Values are strings. The environment holds only text, and so does os.environ. Converting "9000" into 9000 is your code's job, and that conversion is often exactly what is worth testing.

Everything is undone — prove it

The claim is that monkeypatch restores what it changed when the test finishes. Here is the evidence:

python
import os


def test_first(monkeypatch):
    monkeypatch.setenv("APP_DEBUG", "1")
    print("inside first :", os.environ.get("APP_DEBUG"))


def test_second():
    print("inside second:", os.environ.get("APP_DEBUG"))

pytest -q -s (-s lets print through):

text
inside first : 1
.inside second: None
.
2 passed in 0.01s

The second test did not ask for monkeypatch and still sees nothing — the variable is gone. Compare what happens when you write to os.environ directly:

python
import os


def test_first():
    os.environ["APP_DEBUG"] = "1"
    print("inside first :", os.environ.get("APP_DEBUG"))


def test_second():
    print("inside second:", os.environ.get("APP_DEBUG"))
text
inside first : 1
.inside second: 1
.
2 passed in 0.01s

The value has leaked into the next test, and into every test after it in the same run. This is the kind of bug that makes a test pass when run alone and fail when run with the others — or worse, the other way round.

There is no magic in the undo. Before each change, monkeypatch records the old value — or the fact that there was none — and during teardown it puts every recorded value back, in reverse order. Teardown runs whether the test passed, failed or raised, so a failing assertion halfway through cannot leave the environment dirty. A hand-written os.environ[...] = ... followed by a hand-written clean-up line at the end of the test would be skipped by exactly that failing assertion.

setattr — replace a function for one test

Environment variables are the gentle case. The harder one is code that talks to the network:

weather.py:

python
import json
import urllib.request


def fetch_temperature(city):
    url = f"https://api.example.com/weather?city={city}"
    with urllib.request.urlopen(url, timeout=5) as response:
        return json.load(response)["temp_c"]


def advice(city):
    temp = fetch_temperature(city)
    if temp >= 30:
        return "Hot: carry water"
    if temp <= 10:
        return "Cold: take a jacket"
    return "Mild"

The interesting logic is in advice: the thresholds, the messages. Test it as it is and the test depends on a server, a network connection and today's weather. On a machine with no network:

text
$ pytest -q --tb=line test_real.py
F                                                                        [100%]
=================================== FAILURES ===================================
E   socket.gaierror: [Errno -5] No address associated with hostname

During handling of the above exception, another exception occurred:
E   urllib.error.URLError: <urlopen error [Errno -5] No address associated with hostname>
/usr/lib/python3.12/urllib/request.py:1347: urllib.error.URLError: <urlopen error [Errno -5] No address associated with hostname>
1 failed in 0.01s

monkeypatch.setattr swaps fetch_temperature for something you control, for the length of one test:

python
import weather


def test_hot_day(monkeypatch):
    monkeypatch.setattr(weather, "fetch_temperature", lambda city: 35)
    assert weather.advice("Dhaka") == "Hot: carry water"


def test_cold_day(monkeypatch):
    monkeypatch.setattr("weather.fetch_temperature", lambda city: 4)
    assert weather.advice("Oslo") == "Cold: take a jacket"
text
..                                                                       [100%]
2 passed in 0.01s

There are two ways to name the target, and they do the same thing:

  • setattr(weather, "fetch_temperature", new) — the object, then the attribute name as a string.
  • setattr("weather.fetch_temperature", new) — one dotted string; pytest imports weather and splits off the last part.

Use the object form when the test already imports the module; use the string form when it does not need to. Either way, notice what is being tested now: not fetch_temperature, which we replaced, but the thresholds and messages in advice, which is the code we wrote. No network, no waiting, the same answer every time — and a temperature of 35 or 4 on demand, which the real weather would never give you on cue.

Why setattr and not just weather.fetch_temperature = lambda city: 35 in the test? For the same reason as with os.environ: the plain assignment is never undone, and every later test in the run would get the fake.

The rule: patch where it is looked up

Now the mistake that everyone makes once. A small module that rolls a die:

dice.py:

python
from random import randint


def roll():
    return randint(1, 6)


def is_jackpot():
    return roll() == 6

To test the jackpot, force randint to return 6. It lives in random, so patch it there:

python
import random

import dice


def test_jackpot(monkeypatch):
    monkeypatch.setattr(random, "randint", lambda a, b: 6)
    assert dice.is_jackpot() is True
text
F                                                                        [100%]
=================================== FAILURES ===================================
_________________________________ test_jackpot _________________________________

monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fc1b2fdc9b0>

    def test_jackpot(monkeypatch):
        monkeypatch.setattr(random, "randint", lambda a, b: 6)
>       assert dice.is_jackpot() is True
E       assert False is True
E        +  where False = <function is_jackpot at 0x7fc1b2fcd800>()
E        +    where <function is_jackpot at 0x7fc1b2fcd800> = dice.is_jackpot

test_dice.py:8: AssertionError
=========================== short test summary info ============================
FAILED test_dice.py::test_jackpot - assert False is True
1 failed in 0.01s

The patch was applied, and dice never saw it. Run the test a few more times and it will sometimes pass — whenever the real randint happens to roll a 6. A test that is wrong and occasionally green is worse than one that is always red.

The reason is what from random import randint actually does. It does not create a link to random.randint. It copies the reference into a new name, dice.randint, at import time. After that there are two names pointing at the same function, and setattr moves only one of them:

python
import random

import dice

print(dice.randint is random.randint)
random.randint = lambda a, b: 6
print(dice.randint is random.randint)
text
True
False

When roll() runs, Python looks up randint in the module dice — that is where the name is looked up. So that is where it has to be replaced:

python
import dice


def test_jackpot(monkeypatch):
    monkeypatch.setattr(dice, "randint", lambda a, b: 6)
    assert dice.is_jackpot() is True


def test_no_jackpot(monkeypatch):
    monkeypatch.setattr("dice.randint", lambda a, b: 3)
    assert dice.is_jackpot() is False
text
..                                                                       [100%]
2 passed in 0.01s

Five runs in a row, five passes. The other fix is to change how dice.py imports. With import random and a call to random.randint(1, 6), the lookup happens in the module random on every call, and patching random.randint then works:

python
import random


def roll():
    return random.randint(1, 6)


def is_jackpot():
    return roll() == 6
text
$ pytest -q        # the original test, patching random.randint
.                                                                        [100%]
1 passed in 0.01s

The rule in one sentence: find the line that calls the function, see which module it gets the name from, and patch the name there.

setitem — one key of a dictionary

Settings are often kept in a module-level dictionary. Replacing the whole dictionary would be clumsy; setitem changes one key and restores it afterwards:

shop.py:

python
SETTINGS = {"currency": "BDT", "tax_rate": 0.15}


def price_with_tax(amount):
    total = round(amount * (1 + SETTINGS["tax_rate"]), 2)
    return f"{total} {SETTINGS['currency']}"
python
import shop


def test_tax_free_day(monkeypatch):
    monkeypatch.setitem(shop.SETTINGS, "tax_rate", 0.0)
    assert shop.price_with_tax(100) == "100.0 BDT"


def test_other_currency(monkeypatch):
    monkeypatch.setitem(shop.SETTINGS, "currency", "INR")
    assert shop.price_with_tax(100) == "115.0 INR"


def test_settings_are_back():
    assert shop.SETTINGS == {"currency": "BDT", "tax_rate": 0.15}

pytest -v:

text
test_shop.py::test_tax_free_day PASSED                                   [ 33%]
test_shop.py::test_other_currency PASSED                                 [ 66%]
test_shop.py::test_settings_are_back PASSED                              [100%]

============================== 3 passed in 0.01s ===============================

The third test is the proof again: after two tests each changed a key, the dictionary is exactly as it started. Its partner, monkeypatch.delitem(d, key), removes a key for one test.

chdir and syspath_prepend

Code that opens a relative path like "notes.txt" depends on the directory it runs from. Combine chdir with tmp_path from the last chapter:

notes.py:

python
from pathlib import Path


def read_notes():
    return Path("notes.txt").read_text().splitlines()
python
from pathlib import Path

from notes import read_notes


def test_reads_from_current_directory(tmp_path, monkeypatch):
    (tmp_path / "notes.txt").write_text("buy milk\ncall Rina\n")
    monkeypatch.chdir(tmp_path)
    assert Path.cwd() == tmp_path
    assert read_notes() == ["buy milk", "call Rina"]


def test_import_from_a_new_folder(tmp_path, monkeypatch):
    (tmp_path / "helper.py").write_text("ANSWER = 42\n")
    monkeypatch.syspath_prepend(tmp_path)
    import helper
    assert helper.ANSWER == 42
text
..                                                                       [100%]
2 passed in 0.01s

chdir moves the working directory for one test and moves it back afterwards. The second test shows syspath_prepend in its one line of use: it puts a folder at the front of sys.path so a module written there can be imported, and removes it again at the end — handy when testing code that loads plugins from a folder.

Better still: pass the dependency in

Patching works, but every patch is a small admission that the code reached out and grabbed something from the world. Some things cannot be patched comfortably at all. datetime.now is one:

text
E       TypeError: cannot set 'now' attribute of immutable type 'datetime.datetime'

The alternative is a design choice: let the function receive the thing it depends on, with the real one as the default:

shop_hours.py:

python
from datetime import datetime


def is_open(now=datetime.now):
    hour = now().hour
    return 9 <= hour < 21

Production code calls is_open() and gets the real clock. A test passes its own clock:

python
from datetime import datetime

from shop_hours import is_open


def test_closed_just_before_nine():
    assert is_open(now=lambda: datetime(2026, 1, 5, 8, 59)) is False


def test_open_at_nine():
    assert is_open(now=lambda: datetime(2026, 1, 5, 9, 0)) is True


def test_closed_at_nine_pm():
    assert is_open(now=lambda: datetime(2026, 1, 5, 21, 0)) is False
text
...                                                                      [100%]
3 passed in 0.01s

No fixture, nothing to undo, no question of where a name is looked up. The test reads as a plain sentence — "at 8:59 the shop is closed" — and the boundary minutes that matter are easy to hit exactly.

The same shape works for randomness: def pick_winner(customers, choose=random.choice), and a test passes choose=lambda items: items[0]. This is called dependency injection, and it needs no framework: a default argument is enough. Use it when you write the code. Reach for monkeypatch when you are testing code you cannot or should not change, or when the dependency is buried several calls deep.

A complete example

This is the configuration loader planned in "Before you write the test": three layers — built-in defaults, an optional app.cfg file in the current directory, and environment variables that override both. Each row of the plan becomes one test.

app_config.py:

python
import os
from pathlib import Path

DEFAULTS = {"port": "8000", "debug": "0", "region": "bd"}


def read_file(name="app.cfg"):
    path = Path(name)
    if not path.exists():
        return {}
    values = {}
    for line in path.read_text().splitlines():
        if "=" in line:
            key, value = line.split("=", 1)
            values[key.strip()] = value.strip()
    return values


def load_config():
    # Lowest priority first: defaults, then the file, then the environment.
    raw = dict(DEFAULTS)
    raw.update(read_file())
    for key in raw:
        env_name = "APP_" + key.upper()
        if env_name in os.environ:
            raw[key] = os.environ[env_name]

    port = int(raw["port"])
    if not 1 <= port <= 65535:
        raise ValueError(f"port out of range: {port}")
    return {"port": port, "debug": raw["debug"] == "1", "region": raw["region"]}

test_app_config.py:

python
import pytest

import app_config


@pytest.fixture
def clean_env(tmp_path, monkeypatch):
    # An empty folder and no APP_ variables: the test starts from nothing.
    monkeypatch.chdir(tmp_path)
    for name in ("APP_PORT", "APP_DEBUG", "APP_REGION"):
        monkeypatch.delenv(name, raising=False)
    return tmp_path


def test_defaults(clean_env):
    assert app_config.load_config() == {"port": 8000, "debug": False, "region": "bd"}


def test_file_beats_defaults(clean_env):
    (clean_env / "app.cfg").write_text("port = 9000\nregion = in\n")
    config = app_config.load_config()
    assert config["port"] == 9000
    assert config["region"] == "in"


def test_env_beats_file(clean_env, monkeypatch):
    (clean_env / "app.cfg").write_text("port = 9000\n")
    monkeypatch.setenv("APP_PORT", "7000")
    assert app_config.load_config()["port"] == 7000


def test_changed_default(clean_env, monkeypatch):
    monkeypatch.setitem(app_config.DEFAULTS, "region", "ae")
    assert app_config.load_config()["region"] == "ae"


def test_bad_port(clean_env, monkeypatch):
    monkeypatch.setenv("APP_PORT", "70000")
    with pytest.raises(ValueError, match="out of range"):
        app_config.load_config()


def test_reader_can_be_replaced(clean_env, monkeypatch):
    monkeypatch.setattr(app_config, "read_file", lambda: {"port": "1234"})
    assert app_config.load_config()["port"] == 1234

pytest -v:

text
collected 6 items

test_app_config.py::test_defaults PASSED                                 [ 16%]
test_app_config.py::test_file_beats_defaults PASSED                      [ 33%]
test_app_config.py::test_env_beats_file PASSED                           [ 50%]
test_app_config.py::test_changed_default PASSED                          [ 66%]
test_app_config.py::test_bad_port PASSED                                 [ 83%]
test_app_config.py::test_reader_can_be_replaced PASSED                   [100%]

============================== 6 passed in 0.01s ===============================

Three things are worth noticing.

First, the clean_env fixture uses monkeypatch itself. A fixture and the test that requests it share one monkeypatch object, so the test can add its own setenv on top, and every change from both is undone together at the end.

Second, the fixture removes the variables instead of trusting that they are absent. That is what makes the suite give the same answer with APP_PORT=5 APP_REGION=xx pytest -q as without — try it; it still says 6 passed. That is the last row of the plan, satisfied by design rather than by a test of its own.

Third, the last test patches app_config.read_file — the module where load_config looks the name up. Because both functions live in the same module, that is the one place that matters.

When it breaks

AttributeError: <module 'weather' from '.../weather.py'> has no attribute 'fetch_temprature' A typo in the attribute name. setattr refuses to create an attribute that does not already exist, which protects you from "patching" a name nobody ever reads. The string form says it differently — AttributeError: 'module' object at weather has no attribute 'fetch_temprature' — but means the same. Pass raising=False only if you really mean to add a new attribute.

The patch is applied but the real function still runs No error at all: the test fails with the real value, or passes by luck. You patched the name where it is defined (random.randint) instead of where it is looked up (dice.randint). Look at the import line in the code under test.

KeyError: 'APP_DEBUG' from monkeypatch.delenv The variable was not set, and delenv raises by default. Add raising=False. The same applies to delitem and delattr.

PytestWarning: Value of environment variable APP_PORT type should be str, but got 9000 (type: int); converted to str implicitly You wrote setenv("APP_PORT", 9000). Environment variables are text; write "9000".

TypeError: cannot set 'now' attribute of immutable type 'datetime.datetime' Built-in types such as datetime.datetime cannot have their methods replaced. Pass the clock in as an argument instead, as in the is_open example.

ScopeMismatch: You tried to access the function scoped fixture monkeypatch with a module scoped request object. A fixture with scope="module" (or "session") asked for monkeypatch, which lives for one test only. Keep the fixture at function scope, or use pytest.MonkeyPatch.context() inside the wider fixture and manage the undo yourself.