Fixtures, parametrization and temporary files (Python)

Once a suite has thirty tests, the same three lines of set-up start to appear in all of them, and the same input table gets copied into five test functions. pytest has a feature for each problem: fixtures for shared set-up, and parametrization for repeated cases. Both build on the plain-function tests from the previous lesson.

A fixture is a function that supplies a value

Mark a function with @pytest.fixture. Any test that lists a parameter with the same name receives the fixture’s return value. There is no import and no call: pytest matches the names.

import pytest

from bank import Account, InsufficientFunds


@pytest.fixture
def account():
    """An account with 100 in it, rebuilt for every test."""
    return Account("Ana", 100)


@pytest.mark.parametrize(("amount", "expected_balance"), [(10, 90), (100, 0)])
def test_valid_withdrawals(account, amount, expected_balance):
    account.withdraw(amount)
    assert account.balance == expected_balance


@pytest.mark.parametrize(
    ("amount", "error"),
    [(150, InsufficientFunds), (0, ValueError), (-5, ValueError)],
)
def test_refused_withdrawals_leave_the_balance_alone(account, amount, error):
    with pytest.raises(error):
        account.withdraw(amount)
    assert account.balance == 100

This file uses both features, and the next two sections take them one at a time. For the fixture: account is created fresh for each test that asks for it, so a test that changes the balance cannot affect another. That is the same promise setUp gave you in unittest, but a test only gets the fixtures it names.

A fixture defined in the test file is visible only there. To share it, move it into a file called conftest.py in the same folder or a parent. pytest loads it automatically, and the tests need no import. Here are two fixtures for a settings loader:

import json

import pytest


@pytest.fixture
def settings_file(tmp_path):
    """A settings file on disk with one custom value."""
    path = tmp_path / "settings.json"
    path.write_text(json.dumps({"theme": "dark"}), encoding="utf-8")
    return path


@pytest.fixture
def events():
    """A list that records set-up and tear-down, to show the fixture order."""
    log = ["setup"]
    yield log
    log.append("teardown")
    assert log == ["setup", "test body", "teardown"]

Two things are new here. settings_file itself asks for another fixture, tmp_path, which pytest provides. Fixtures can depend on fixtures. And events uses yield instead of return. Everything before the yield is set-up, the yielded value goes to the test, and everything after it runs when the test is over, whether it passed or not. That is the place to close a connection or remove a file. If you have read context managers, the shape will look familiar.

The code being tested is a settings loader whose folder can be moved with an environment variable:

"""Load user settings from a JSON file whose folder can be set by APP_HOME."""

import json
import os
from pathlib import Path

DEFAULTS = {"theme": "light", "font_size": 12}


def settings_path():
    home = os.environ.get("APP_HOME")
    base = Path(home) if home else Path.home() / ".app"
    return base / "settings.json"


def load_settings(path=None):
    path = Path(path) if path else settings_path()
    if not path.exists():
        return dict(DEFAULTS)
    return {**DEFAULTS, **json.loads(path.read_text(encoding="utf-8"))}

tmp_path and monkeypatch keep tests away from your real files

Tests that touch the real home folder or the real environment are fragile, and they can do damage. pytest ships two fixtures that remove the risk.

tmp_path is a pathlib.Path to a fresh empty folder, unique to the test. See pathlib for what you can do with it. pytest keeps the last few runs’ folders around for inspection and deletes older ones.
monkeypatch changes things for the duration of one test and puts them back afterwards: setenv, delenv, setattr, chdir.

from pathlib import Path

from appconfig import DEFAULTS, load_settings, settings_path


def test_missing_file_gives_the_defaults(tmp_path):
    assert load_settings(tmp_path / "nothing-here.json") == DEFAULTS


def test_file_values_override_defaults(settings_file):
    settings = load_settings(settings_file)
    assert settings["theme"] == "dark"
    assert settings["font_size"] == 12


def test_settings_path_follows_app_home(monkeypatch, tmp_path):
    monkeypatch.setenv("APP_HOME", str(tmp_path))
    assert settings_path() == tmp_path / "settings.json"

The next three tests show the environment and the yield fixture:

def test_settings_path_falls_back_to_the_home_folder(monkeypatch):
    monkeypatch.delenv("APP_HOME", raising=False)
    assert settings_path() == Path.home() / ".app" / "settings.json"


def test_load_settings_uses_app_home(monkeypatch, settings_file):
    monkeypatch.setenv("APP_HOME", str(settings_file.parent))
    assert load_settings()["theme"] == "dark"


def test_fixture_teardown_runs_after_the_test(events):
    events.append("test body")
    assert events == ["setup", "test body"]

The fallback test calls delenv(..., raising=False) so that it passes whether or not the variable was set on your machine. After each test, the environment is exactly as it was. The last test shows the yield fixture at work: the assert inside the fixture after the yield checks that tear-down ran last.

Other built-in fixtures worth knowing are capsys (captures what the code prints, so you can assert on it), caplog (captures log records, see logging) and tmp_path_factory (a temp folder shared across tests). Run python -m pytest --fixtures to list every fixture available.

One test, many inputs

The tests above for settings have one case each. When the same check applies to a table of inputs, @pytest.mark.parametrize runs the function once per row. The first argument names the parameters, and the second is a list of tuples of values.

"""Parse numbers typed by people: spaces, commas and underscores."""


def parse_number(text):
    """Return an int or a float for text such as ' 1,234 ' or '3.5'."""
    cleaned = text.strip().replace(",", "").replace("_", "")
    if not cleaned:
        raise ValueError("no number in empty text")
    try:
        return int(cleaned)
    except ValueError:
        return float(cleaned)

import pytest

from numparse import parse_number


@pytest.mark.parametrize(
    ("text", "expected"),
    [
        ("42", 42),
        ("  42  ", 42),
        ("1,234", 1234),
        ("1_000_000", 1_000_000),
        ("3.5", 3.5),
        ("-7", -7),
        ("1e3", 1000.0),
    ],
)
def test_parses_valid_text(text, expected):
    result = parse_number(text)
    assert result == expected
    assert type(result) is type(expected)

A table of inputs that must fail needs only one parameter per row. Rows can also carry a readable id:

@pytest.mark.parametrize("text", ["", "   ", "abc", "1.2.3", "12 34"])
def test_rejects_invalid_text(text):
    with pytest.raises(ValueError):
        parse_number(text)


@pytest.mark.parametrize(
    ("text", "expected"),
    [
        pytest.param("0.1", 0.1, id="simple-float"),
        pytest.param("1,000.5", 1000.5, id="thousands-and-decimal"),
    ],
)
def test_named_cases(text, expected):
    assert parse_number(text) == expected

Each row becomes its own test with an id built from the values, so a failure names the row:

$ python -m pytest -v --no-header tests/test_numparse.py
============================= test session starts ==============================
collecting ... collected 14 items

tests/test_numparse.py::test_parses_valid_text[42-42] PASSED             [  7%]
tests/test_numparse.py::test_parses_valid_text[  42  -42] PASSED         [ 14%]
tests/test_numparse.py::test_parses_valid_text[1,234-1234] PASSED        [ 21%]
tests/test_numparse.py::test_parses_valid_text[1_000_000-1000000] PASSED [ 28%]
tests/test_numparse.py::test_parses_valid_text[3.5-3.5] PASSED           [ 35%]
tests/test_numparse.py::test_parses_valid_text[-7--7] PASSED             [ 42%]
tests/test_numparse.py::test_parses_valid_text[1e3-1000.0] PASSED        [ 50%]
tests/test_numparse.py::test_rejects_invalid_text[] PASSED               [ 57%]
tests/test_numparse.py::test_rejects_invalid_text[   ] PASSED            [ 64%]
tests/test_numparse.py::test_rejects_invalid_text[abc] PASSED            [ 71%]
tests/test_numparse.py::test_rejects_invalid_text[1.2.3] PASSED          [ 78%]
tests/test_numparse.py::test_rejects_invalid_text[12 34] PASSED          [ 85%]
tests/test_numparse.py::test_named_cases[simple-float] PASSED            [ 92%]
tests/test_numparse.py::test_named_cases[thousands-and-decimal] PASSED   [100%]

============================== 14 passed in 0.02s ==============================

Where the generated id would be unreadable, pytest.param(..., id="name") gives the row a label. Compare this with subTest from the unittest lesson: subTest reports several failures inside one test, while parametrization produces independent tests that you can select with -k and re-run one by one.

Common pitfalls

Mutable data shared between tests. If a fixture returns a module-level list, every test mutates the same object. Create the value inside the fixture function.
Wider fixture scope than you need. @pytest.fixture(scope="module") builds the value once for the whole file, which is faster for something costly like a database connection, but state then leaks between tests. Keep the default (one per test) until a profiler says otherwise.
A typo in a fixture name. A parameter that matches no fixture gives fixture 'acount' not found at set-up. The error lists the fixtures that are available.
Using monkeypatch on os.environ by hand. Direct changes survive the test and break the next one. Use setenv and delenv.
Giant parameter tables with hidden logic. If the expected value is computed with the same code as the function, the test proves nothing. Write expectations out literally.
Parametrizing with a mutable default. Each row’s objects are shared if you reuse the same list object in several rows. Construct them separately.

ExerciseTurn a loop into a parametrized test

This test checks two slugs with a for loop, and stops at the first failure. Rewrite it as one parametrized test over five pairs: ("Hello World", "hello-world"), (" padded ", "padded"), ("Rock & Roll", "rock-roll"), ("a.b.c", "a-b-c") and ("", ""). Run it: the last line should read 5 passed.

from textutil import slugify


def test_slugify_table():
    cases = [("Hello World", "hello-world"), ("  padded  ", "padded")]
    for text, expected in cases:
        assert slugify(text) == expected

Show solution

import pytest

from textutil import slugify


@pytest.mark.parametrize(
    ("text", "expected"),
    [
        ("Hello World", "hello-world"),
        ("  padded  ", "padded"),
        ("Rock & Roll", "rock-roll"),
        ("a.b.c", "a-b-c"),
        ("", ""),
    ],
)
def test_slugify_table(text, expected):
    assert slugify(text) == expected

With -v, the ids read test_slugify_table[Hello World-hello-world], test_slugify_table[ padded -padded] and so on, and the empty row appears as [-]. Five rows give five tests, and a failing row no longer hides the ones after it.

ExerciseTest a notebook with tmp_path

Here is a module that saves and loads notes as text files. Write tests for it that never touch your real folders. One parametrized test should save then load the texts "", "hello" and "two\nlines" and check each comes back unchanged. A second test should check that loading a note that does not exist raises FileNotFoundError with no note called 'plans'. Run them: the last line should read 4 passed.

"""Save and load short notes as text files, one file per title."""

from pathlib import Path


def save_note(folder, title, text):
    """Write text to <folder>/<title>.txt and return that path."""
    path = Path(folder) / f"{title}.txt"
    path.write_text(text, encoding="utf-8")
    return path


def load_note(folder, title):
    """Return the text of a saved note, or raise FileNotFoundError."""
    path = Path(folder) / f"{title}.txt"
    if not path.exists():
        raise FileNotFoundError(f"no note called {title!r}")
    return path.read_text(encoding="utf-8")

Show solution

tmp_path gives each test its own empty folder. It is passed to save_note as the folder, and parametrize supplies text:

import pytest

from notebook import load_note, save_note


@pytest.mark.parametrize("text", ["", "hello", "two\nlines"])
def test_a_saved_note_reads_back_unchanged(tmp_path, text):
    save_note(tmp_path, "draft", text)
    assert load_note(tmp_path, "draft") == text


def test_a_missing_note_is_reported_by_name(tmp_path):
    with pytest.raises(FileNotFoundError, match="no note called 'plans'"):
        load_note(tmp_path, "plans")

The three rows make three tests and the missing-note test makes the fourth. Because tmp_path is new for every test, no note saved by one test can be found by another.

ExercisePredict how many tests run

Two parametrize decorators are stacked on one test. Before running it, predict how many tests pytest reports, and the ids of the first and last.

import pytest


@pytest.mark.parametrize("quantity", [1, 2, 3])
@pytest.mark.parametrize("price", [10, 20])
def test_order_total_is_positive(quantity, price):
    assert quantity * price > 0

Show solution

Stacked decorators multiply: 2 prices times 3 quantities makes 6 tests, and python -m pytest -v ends with 6 passed. The decorator nearest the function varies slowest in the id, so the ids run from test_order_total_is_positive[10-1] to test_order_total_is_positive[20-3], with price first and quantity second. One table per decorator is often clearer than one long list of every combination.

Summary

A fixture is a function marked @pytest.fixture. A test receives it by naming a parameter after it, and conftest.py shares fixtures across files. Code after a yield runs as tear-down.
tmp_path gives each test an empty folder, and monkeypatch changes environment variables, attributes or the working folder and undoes the change afterwards.
@pytest.mark.parametrize runs one test over a table, giving each row its own result and id. pytest.param(..., id=...) names a row.
The next lesson puts tests first: a workflow where the failing test comes before the code.

A fixture is a function that supplies a value

Sharing fixtures with conftest.py

tmp_path and monkeypatch keep tests away from your real files

One test, many inputs

Common pitfalls

Summary