Debugging habits

The hardest bugs are not the ones that crash. A crash comes with a traceback that points at a line. The hard ones run to the end and print the wrong answer. Nothing tells you where to look, so you need a method. This lesson gives you a small routine and two tools: print used with care, and the pdb debugger.

A routine that works

1. Reproduce it. Find the smallest input that gives the wrong answer, and write down what you expected.
2. Guess where it goes wrong. Pick the first place where the program could have a wrong value, not the last place it shows.
3. Look at the real values. Do not trust your picture of the program. Print the variable or inspect it.
4. Change one thing at a time, then run again.
5. Explain it aloud. If you are stuck, describe the code line by line to a friend, or to a rubber duck on your desk. Saying “this line adds the tax to the total” while reading the line total * 1 + rate is often enough to hear the mistake.

Here is a function that should add tax to a total. With prices 10 and 20 and a tax rate of 0.1 (10 per cent), the answer should be 33.0.

def total_with_tax(prices, rate):
    total = 0
    for price in prices:
        total += price
    return total * 1 + rate


print(total_with_tax([10, 20], 0.1))

30.1

The program runs and prints a plausible number that is wrong. The expected value was 33.0, so the bug is somewhere between the input and the return.

Print debugging with {x=}

The oldest debugging tool is also the simplest: print the values. The f-string form f"{total=}" prints the name and the value together, so you do not have to type the label yourself. It comes from the f-strings lesson.

def total_with_tax(prices, rate):
    total = 0
    for price in prices:
        total += price
    print(f"{total=} {rate=}")
    result = total * 1 + rate
    print(f"{result=}")
    return result


print(total_with_tax([10, 20], 0.1))

total=30 rate=0.1
result=30.1
30.1

The last line is the returned value, printed by the call. The first line shows that total and rate are correct, so the loop is fine. The second shows that the problem is in the line between them: total * 1 + rate adds 0.1 to 30 instead of multiplying by 1.1. The fix is to put the addition in brackets:

def total_with_tax(prices, rate):
    total = 0
    for price in prices:
        total += price
    return round(total * (1 + rate), 2)


print(total_with_tax([10, 20], 0.1))

33.0

The two prints were a way of asking “is it here yet?” and moving closer. Remove them when you are done, or the final program is full of noise.

Seeing invisible differences with !r

Plain print shows what a string looks like. !r (or repr) shows what it is, with quotes, so spaces and special characters become visible.

answer = "yes "
print(answer == "yes")
print("plain print:", answer, "|")
print(f"{answer=}")
print(f"{answer!r}")
print(f"{len(answer)=}")

False
plain print: yes  |
answer='yes '
'yes '
len(answer)=4

The comparison is False and plain print gives no clue why. The {answer=} form shows the quotes, and a trailing space becomes obvious. len confirms four characters rather than three. The cure is in the string methods lesson: answer.strip().

Stopping the program with breakpoint()

Print debugging has a cost: add a print, run, read, add another. The alternative is to pause the program in the middle and look around. Call the built-in breakpoint() where you want to stop. When Python reaches it, it starts the debugger, pdb, and waits for your commands.

def total_with_tax(prices, rate):
    total = 0
    for price in prices:
        total += price
    breakpoint()
    return total * 1 + rate


print(total_with_tax([10, 20], 0.1))

Run the file as usual. The program stops and shows a (Pdb) prompt. Type a command after it. Here is a session, with the commands you type after each prompt. The path on your machine will differ.

> /home/you/project/pdb_demo.py(5)total_with_tax()
-> breakpoint()
(Pdb) p total
30
(Pdb) p rate
0.1
(Pdb) n
> /home/you/project/pdb_demo.py(6)total_with_tax()
-> return total * 1 + rate
(Pdb) p total * 1 + rate
30.1
(Pdb) c
30.1

The first two lines say which file, line and function you are in, and which line is about to run (the one with ->). Then p total printed 30, so the loop did its job. n (next) ran one line and showed the next. The expression p total * 1 + rate produced the wrong 30.1, which points straight at the bug. c (continue) let the program run to the end.

The commands you need most:

You can also type any Python statement at the prompt. If the name collides with a command, such as a variable called n or c, write p n or !n.

You can also start a script under the debugger from the beginning with python -m pdb script.py. The intermediate level goes deeper into pdb, post-mortem debugging and the other tools.

Print or debugger?

Use print when you know roughly where the problem is and want a quick look, or when the program runs somewhere you cannot pause, such as a script started by another tool. Use breakpoint() when you do not know which value is wrong, because one pause lets you look at many values and try expressions without re-running. Most programmers use both.

Common pitfalls

• Printing the wrong thing. A print placed after the bug tells you nothing. Print before and after the suspect line.
• Changing several things at once. If the output changes, you do not know which change did it.
• Trusting what print shows for strings. Use {x=} or !r to see spaces and quotes.
• Leaving debug prints or breakpoint() in finished code.
• Typing a variable name that is also a pdb command. p name always prints the variable.
• Debugging a different bug than the one you have. Re-read the expected result. Sometimes the program is right and the expectation is wrong.

def average(numbers):
    total = 0
    for n in numbers:
        total = n
    return total / len(numbers)


print(average([2, 4, 6]))
print(average([5]))

4.0
5.0

def average(numbers):
    total = 0
    for n in numbers:
        total += n
    return total / len(numbers)


print(average([2, 4, 6]))
print(average([5]))

4.0
5.0

answer = "yes "
if answer == "yes":
    print("Confirmed")
else:
    print("Cancelled")

answer = "yes "
if answer.strip().lower() == "yes":
    print("Confirmed")
else:
    print("Cancelled")

Confirmed

(Pdb) p prices
[10, 20]
(Pdb) p sum(prices) * (1 + rate)
33.0
(Pdb) c
30.1

Summary

• Reproduce the bug with the smallest input, look at the real values, and change one thing at a time.
• print(f"{x=}") shows a name with its value, and !r shows quotes so hidden spaces appear.
• breakpoint() pauses the program under pdb. Use p to print, n for the next line, s to step into a call, c to continue and q to quit.
• Remove debug prints and breakpoint() when the bug is fixed.

Next: mini-project, a word-frequency counter.