Automated tests with unittest

Checking your code by running it and looking at the output works until the program has twenty functions. A test turns that look into something the computer repeats for you: a call, the answer you expect, and a verdict. If the testing mindset is new, the fundamentals lesson on it explains what to test and why. This lesson is about Python’s built-in tool for writing those checks, unittest.

Here is the code we will test. It lives in c32_textutil.py, and nothing in it knows that tests exist.

"""Small text helpers used as the code under test."""


def slugify(text):
    """Turn a title into a lowercase, hyphen-separated slug."""
    words = "".join(ch.lower() if ch.isalnum() else " " for ch in text).split()
    return "-".join(words)


def average(numbers):
    """Return the mean of a non-empty sequence of numbers."""
    if not numbers:
        raise ValueError("average() needs at least one number")
    return sum(numbers) / len(numbers)

A test case is a class with test methods

A unittest test lives in a subclass of unittest.TestCase. Every method whose name starts with test is one test. Inside it, you call the code and check the result with an assert method.

import unittest

from c32_textutil import average, slugify


class SlugifyTests(unittest.TestCase):
    def test_lowercases_and_joins_with_hyphens(self):
        self.assertEqual(slugify("Hello World"), "hello-world")

    def test_drops_punctuation(self):
        self.assertEqual(slugify("Hello, World!"), "hello-world")

    def test_empty_text_gives_empty_slug(self):
        self.assertEqual(slugify(""), "")

Each test checks one behaviour, and its name says which. The expected value is written literally, not computed with the same logic as the code. The second class tests average, including the error case:

class AverageTests(unittest.TestCase):
    def test_mean_of_integers(self):
        self.assertEqual(average([2, 4, 6]), 4)

    def test_float_result_needs_tolerance(self):
        self.assertAlmostEqual(average([0.1, 0.2]), 0.15)

    def test_empty_list_is_an_error(self):
        with self.assertRaises(ValueError) as caught:
            average([])
        self.assertIn("at least one", str(caught.exception))


if __name__ == "__main__":
    unittest.main()

assertAlmostEqual compares floats within a tolerance, because 0.1 + 0.2 is not exactly 0.3 in binary floating point, as the numbers lesson showed. assertRaises used as a with block passes only if the block raises that exception. Binding it with as caught lets you inspect the exception afterwards through caught.exception.

Running the tests

The last two lines let you run the file as a script. The more common way is to let unittest find and run the tests from the command line. The -v flag prints one line per test:

$ python -m unittest -v c32_test_textutil.py
test_empty_list_is_an_error (c32_test_textutil.AverageTests.test_empty_list_is_an_error) ... ok
test_float_result_needs_tolerance (c32_test_textutil.AverageTests.test_float_result_needs_tolerance) ... ok
test_mean_of_integers (c32_test_textutil.AverageTests.test_mean_of_integers) ... ok
test_drops_punctuation (c32_test_textutil.SlugifyTests.test_drops_punctuation) ... ok
test_empty_text_gives_empty_slug (c32_test_textutil.SlugifyTests.test_empty_text_gives_empty_slug) ... ok
test_lowercases_and_joins_with_hyphens (c32_test_textutil.SlugifyTests.test_lowercases_and_joins_with_hyphens) ... ok

----------------------------------------------------------------------
Ran 6 tests in 0.001s

OK

Without -v, each test prints one character: . for a pass, F for a failure and E for an error. Give a dotted name such as c32_test_textutil.AverageTests.test_empty_list_is_an_error to run a single test. With no file name, python -m unittest discovers files named test*.py in the current folder and below, so a project can run everything with one command.

Run order is alphabetical by method name, not the order in the file. Tests must not depend on each other. If one only passes after another has run, the bug is in the tests.

Reading a failure

A test can end in three ways. It passes (ok), it fails because an assertion was false (FAIL), or it errors because something else went wrong, such as an exception nobody expected (ERROR). The difference tells you where to look. Here are three tests written to disagree with the code:

import unittest

from c32_textutil import slugify


class SlugifyTests(unittest.TestCase):
    def test_plus_signs_kept(self):
        self.assertEqual(slugify("C++ tips"), "c++-tips")

    def test_numbers_kept(self):
        self.assertEqual(slugify("Top 10 tips"), "top-10-tips")

    def test_none_is_rejected_politely(self):
        self.assertEqual(slugify(None), "")


if __name__ == "__main__":
    unittest.main()

$ python -m unittest -v c32_failing.py
test_none_is_rejected_politely (c32_failing.SlugifyTests.test_none_is_rejected_politely) ... ERROR
test_numbers_kept (c32_failing.SlugifyTests.test_numbers_kept) ... ok
test_plus_signs_kept (c32_failing.SlugifyTests.test_plus_signs_kept) ... FAIL

======================================================================
ERROR: test_none_is_rejected_politely (c32_failing.SlugifyTests.test_none_is_rejected_politely)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c32_failing.py", line 14, in test_none_is_rejected_politely
    self.assertEqual(slugify(None), "")
                     ~~~~~~~^^^^^^
  File "c32_textutil.py", line 6, in slugify
    words = "".join(ch.lower() if ch.isalnum() else " " for ch in text).split()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable

======================================================================
FAIL: test_plus_signs_kept (c32_failing.SlugifyTests.test_plus_signs_kept)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c32_failing.py", line 8, in test_plus_signs_kept
    self.assertEqual(slugify("C++ tips"), "c++-tips")
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'c-tips' != 'c++-tips'
- c-tips
+ c++-tips
?  ++


----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1, errors=1)

Each problem gets a block after the list of verdicts.

• The FAIL block is a disagreement. The function ran and returned 'c-tips', and the test said 'c++-tips'. assertEqual prints a diff of the two strings, with ? marking the characters that differ. Either the code or the expectation is wrong, and you decide which.
• The ERROR block is a crash. slugify(None) raised TypeError before any comparison happened, and the traceback points into the function. It is the same kind of traceback you learned to read in reading tracebacks. The ~~~^^^ marker lines appear in Python 3.11 and later, and real runs show the full path of each file.

The assert methods you will use most

assertEqual(a, b) covers most cases. The others make intent clearer, and give better failure messages than a bare assertTrue(a == b) would.

Shared preparation with setUp

When several tests need the same starting state, put the preparation in setUp. unittest calls it before every test method, so each test gets its own fresh objects. First, a class to test:

import unittest


class Basket:
    def __init__(self):
        self.items = []

    def add(self, name, price):
        self.items.append((name, price))

    def total(self):
        return sum(price for _, price in self.items)

Then the tests, which all start from a basket that already holds some tea:

class BasketTests(unittest.TestCase):
    def setUp(self):
        # runs before every test method, so each test gets a fresh basket
        self.basket = Basket()
        self.basket.add("tea", 3)

    def test_total_with_one_item(self):
        self.assertEqual(self.basket.total(), 3)

    def test_total_after_adding(self):
        self.basket.add("cake", 4)
        self.assertEqual(self.basket.total(), 7)

    def test_tests_do_not_share_state(self):
        self.assertEqual(len(self.basket.items), 1)


if __name__ == "__main__":
    unittest.main()

$ python -m unittest -v c32_setup.py
test_tests_do_not_share_state (c32_setup.BasketTests.test_tests_do_not_share_state) ... ok
test_total_after_adding (c32_setup.BasketTests.test_total_after_adding) ... ok
test_total_with_one_item (c32_setup.BasketTests.test_total_with_one_item) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.001s

OK

test_total_after_adding appends a second item, and test_tests_do_not_share_state still sees only one. That is what setUp buys you. Its partner tearDown runs after each test, even a failing one, for cleanup such as closing a file. A more flexible tool for the same job is the context manager, which you met in context managers: self.enterContext(...) 3.11+Requires version 3.11 or later enters one and registers its exit as cleanup for the test.

Many cases in one test: subTest

A table of inputs and expected outputs is a good shape for a test, but a plain loop stops at the first failure. subTest runs every row and reports each failing one separately, labelled with the values you pass.

import unittest

from c32_textutil import slugify


class SlugifyTable(unittest.TestCase):
    def test_table(self):
        cases = [
            ("Hello World", "hello-world"),
            ("  padded  ", "padded"),
            ("Rock & Roll", "rock-roll"),
            ("C++ tips", "c-tips"),
            ("v1.2 notes", "v1.2-notes"),
            ("a.b.c", "a.b.c"),
            ("snake_case name", "snake-case-name"),
        ]
        for text, expected in cases:
            with self.subTest(text=text):
                self.assertEqual(slugify(text), expected)


if __name__ == "__main__":
    unittest.main()

$ python -m unittest -v c32_subtest.py
test_table (c32_subtest.SlugifyTable.test_table) ...
  test_table (c32_subtest.SlugifyTable.test_table) (text='v1.2 notes') ... FAIL
  test_table (c32_subtest.SlugifyTable.test_table) (text='a.b.c') ... FAIL

======================================================================
FAIL: test_table (c32_subtest.SlugifyTable.test_table) (text='v1.2 notes')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c32_subtest.py", line 19, in test_table
    self.assertEqual(slugify(text), expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'v1-2-notes' != 'v1.2-notes'
- v1-2-notes
?   ^
+ v1.2-notes
?   ^


======================================================================
FAIL: test_table (c32_subtest.SlugifyTable.test_table) (text='a.b.c')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c32_subtest.py", line 19, in test_table
    self.assertEqual(slugify(text), expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'a-b-c' != 'a.b.c'
- a-b-c
+ a.b.c


----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=2)

The five other rows pass, and the two failing ones are named by their text= value. Without subTest, the loop would have stopped at the first.

Common pitfalls

• A test method name that does not start with test. unittest silently skips check_empty. Look at the count in Ran N tests.
• Tests that share state. A module-level list that one test appends to makes the next test depend on run order. Build fresh objects in setUp.
• assertTrue(a == b). It works, but a failure says only False is not true. assertEqual(a, b) prints both values.
• Comparing floats with assertEqual. Use assertAlmostEqual, or compare with round.
• Testing the implementation, not the behaviour. A test that reaches into private attributes breaks when you refactor correct code. Test through the public functions.
• A test file named like a module you import. A file called test.py or unittest.py shadows standard library modules. Name test files test_<module>.py.

import unittest

def discount(price, percent):
    return price - price * percent / 100


class DiscountTests(unittest.TestCase):
    def test_ten_percent(self):
        self.assertEqual(discount(200, 10), 180)

    def test_zero_percent(self):
        self.assertEqual(discount(50, 0), 50)

    def test_full_discount(self):
        self.assertEqual(discount(80, 100), 80)

    def test_text_price(self):
        self.assertEqual(discount("100", 10), 90)


if __name__ == "__main__":
    unittest.main()

$ python -m unittest -v c32_ex_predict.py
test_full_discount (c32_ex_predict.DiscountTests.test_full_discount) ... FAIL
test_ten_percent (c32_ex_predict.DiscountTests.test_ten_percent) ... ok
test_text_price (c32_ex_predict.DiscountTests.test_text_price) ... ERROR
test_zero_percent (c32_ex_predict.DiscountTests.test_zero_percent) ... ok

======================================================================
ERROR: test_text_price (c32_ex_predict.DiscountTests.test_text_price)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c32_ex_predict.py", line 18, in test_text_price
    self.assertEqual(discount("100", 10), 90)
                     ~~~~~~~~^^^^^^^^^^^
  File "c32_ex_predict.py", line 4, in discount
    return price - price * percent / 100
                   ~~~~~~~~~~~~~~~~^~~~~
TypeError: unsupported operand type(s) for /: 'str' and 'int'

======================================================================
FAIL: test_full_discount (c32_ex_predict.DiscountTests.test_full_discount)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c32_ex_predict.py", line 15, in test_full_discount
    self.assertEqual(discount(80, 100), 80)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 0.0 != 80

----------------------------------------------------------------------
Ran 4 tests in 0.001s

FAILED (failures=1, errors=1)

def is_leap_year(year):
    return year % 4 == 0

import unittest

def is_leap_year(year):
    return year % 4 == 0


class LeapYearTests(unittest.TestCase):
    def test_known_years(self):
        cases = [
            (2024, True),
            (2023, False),
            (1900, False),
            (2000, True),
            (2100, False),
        ]
        for year, expected in cases:
            with self.subTest(year=year):
                self.assertEqual(is_leap_year(year), expected)


if __name__ == "__main__":
    unittest.main()

$ python -m unittest -v c32_ex_leap.py
test_known_years (c32_ex_leap.LeapYearTests.test_known_years) ...
  test_known_years (c32_ex_leap.LeapYearTests.test_known_years) (year=1900) ... FAIL
  test_known_years (c32_ex_leap.LeapYearTests.test_known_years) (year=2100) ... FAIL

======================================================================
FAIL: test_known_years (c32_ex_leap.LeapYearTests.test_known_years) (year=1900)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c32_ex_leap.py", line 18, in test_known_years
    self.assertEqual(is_leap_year(year), expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: True != False

======================================================================
FAIL: test_known_years (c32_ex_leap.LeapYearTests.test_known_years) (year=2100)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c32_ex_leap.py", line 18, in test_known_years
    self.assertEqual(is_leap_year(year), expected)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: True != False

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=2)

import unittest

def is_leap_year(year):
    return year % 4 == 0 and (year % 100 != 0 or year % 400 == 0)


class LeapYearTests(unittest.TestCase):
    def test_known_years(self):
        cases = [
            (2024, True),
            (2023, False),
            (1900, False),
            (2000, True),
            (2100, False),
        ]
        for year, expected in cases:
            with self.subTest(year=year):
                self.assertEqual(is_leap_year(year), expected)


if __name__ == "__main__":
    unittest.main()

$ python -m unittest -v c32_ex_leap_fixed.py
test_known_years (c32_ex_leap_fixed.LeapYearTests.test_known_years) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.001s

OK

import unittest


def clamp(value, low, high):
    return max(low, min(value, high))


class ClampTests(unittest.TestCase):
    def test_inside_range(self):
        self.assertEqual(clamp(5, 0, 10), 5)

    def test_below_range(self):
        self.assertEqual(clamp(-3, 0, 10), 0)

    def check_above_range(self):
        self.assertEqual(clamp(99, 0, 10), 10)


if __name__ == "__main__":
    unittest.main()

$ python -m unittest -v c32_ex_missing_fixed.py
test_above_range (c32_ex_missing_fixed.ClampTests.test_above_range) ... ok
test_below_range (c32_ex_missing_fixed.ClampTests.test_below_range) ... ok
test_inside_range (c32_ex_missing_fixed.ClampTests.test_inside_range) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.001s

OK

Summary

• A unittest test is a method starting with test in a TestCase subclass. Run it with python -m unittest -v file.py, and read FAIL as a disagreement and ERROR as a crash.
• Use assertEqual for most checks, assertAlmostEqual for floats, and assertRaises as a context manager for exceptions.
• setUp gives each test fresh objects, and subTest runs every row of a table and names the failing ones.
• The next lesson shows pytest, which runs the same kind of tests with much less ceremony.