Writing your own modules

The beginner module lesson used modules that someone else wrote. A script that has grown past a couple of hundred lines needs the same treatment: a few files, each with one job, that import each other. This lesson shows what an import does, so that the files you split off behave the way you expect.

A module is a file, and import runs it once

Any .py file can be imported. The module’s name is the file name without .py. Here is a tiny module that announces itself:

print("noisy.py is running")

VALUE = 42

Importing it runs the file from top to bottom, once. Every later import noisy, in this file or any other, reuses the result.

import sys

import noisy
import noisy as again

print(noisy is again)
print("noisy" in sys.modules)
print(noisy.VALUE)

noisy.py is running
True
True
42

“noisy.py is running” appears once, although there are two import lines. The first import did four things in order: found the file, ran it, stored the resulting module object in the dictionary sys.modules, and bound the name noisy. The second import found "noisy" already in sys.modules and simply bound another name to the same object, which is why noisy is again is True.

Two consequences follow. A module’s top-level code is a poor place for anything slow or noisy, because it runs as a side effect of someone else’s import. And editing a module while a program is running does nothing, because the cached copy is used.

One file, two jobs: the __name__ guard

Before running a module’s code, Python sets a variable called __name__ inside it. When the file is imported, __name__ is the module’s name. When the file is run directly, as python temperature.py, __name__ is the string "__main__".

That lets a file carry its own demonstration or command-line behaviour without running it on import:

def to_fahrenheit(celsius):
    return celsius * 9 / 5 + 32


def to_celsius(fahrenheit):
    return (fahrenheit - 32) * 5 / 9


if __name__ == "__main__":
    print("__name__ is", __name__)
    print(to_fahrenheit(100))
    print(to_celsius(212))

Run directly, the guarded block executes:

$ python temperature.py
__name__ is __main__
212.0
100.0

Imported from another file, it does not:

import temperature

print("__name__ in temperature is", temperature.__name__)
print(temperature.to_fahrenheit(37))

__name__ in temperature is temperature
98.6

The functions are defined in both cases. Only the indented block under the if is skipped on import.

Most scripts put their real work in a function called main() and end the file with the guard, as the split program below does. The guard line is then the only code that runs at the top level, and main() can be imported and called from a test.

Where Python looks: sys.path

When you write import temperature, Python checks sys.modules first. If the name is missing, it walks through the list sys.path and looks in each entry for temperature.py or a package folder called temperature. The first match wins.

For a normal program the list starts with the folder of the script you ran, followed by the standard library folders and the site-packages folder where installed packages go. You can check the first entry and ask importlib where a name would be found without importing it:

import os
import sys
from importlib.util import find_spec

here = os.path.dirname(os.path.abspath(__file__))
print(sys.path[0] == here)

spec = find_spec("temperature")
print(spec.origin == os.path.join(here, "temperature.py"))

print(find_spec("no_such_module_anywhere"))

True
True
None

find_spec returns None when nothing matches, which is the case where a real import raises ModuleNotFoundError. When an import finds the wrong file, print(module.__file__) or find_spec(name).origin is the quickest way to see which one it picked.

Because the script’s own folder comes first, a file of yours that shares a name with a standard library module hides the real one. The beginner lesson showed this with random.py.

The first entry is the part that changes with how you launch Python:

• python script.py puts the folder containing script.py first.
• python -m package.module puts the current working directory first.
• python -c "..." and the REPL put an empty string first, which means the current directory.

Circular imports

Suppose a.py starts with from b import hello_b, and b.py starts with from a import hello_a. Importing a begins running a, which imports b, which asks a for a name that has not been defined yet, because a has not got past its first line.

Traceback (most recent call last):
  File "main.py", line 1, in <module>
    import a
  File "a.py", line 1, in <module>
    from b import hello_b
  File "b.py", line 1, in <module>
    from a import hello_a
ImportError: cannot import name 'hello_a' from 'a'

The exact wording of the last line varies between Python versions. The usual cures are to move the shared code into a third module that both files import, or to import the other module inside the one function that needs it. Needing a cycle at all usually means two modules are really one.

Splitting a script

Here is a short script that parses some scores and reports on them. It works, but the parsing, the arithmetic and the printing are tangled in one file.

RAW = """\
ana,82
ben,67
cleo,91
dev,58
"""


def parse(text):
    scores = {}
    for line in text.splitlines():
        name, score = line.split(",")
        scores[name] = int(score)
    return scores


def average(scores):
    return sum(scores.values()) / len(scores)


def best(scores):
    return max(scores, key=scores.get)


scores = parse(RAW)
print(f"Students: {len(scores)}")
print(f"Average: {average(scores):.1f}")
print(f"Top: {best(scores)}")

Students: 4
Average: 74.5
Top: cleo

Moving each job to its own file leaves three small modules. The two library modules contain only definitions:

def parse(text):
    scores = {}
    for line in text.splitlines():
        name, score = line.split(",")
        scores[name] = int(score)
    return scores

def average(scores):
    return sum(scores.values()) / len(scores)


def best(scores):
    return max(scores, key=scores.get)

The entry point imports them and guards its own work:

from parsing import parse
from summary import average, best

RAW = """\
ana,82
ben,67
cleo,91
dev,58
"""


def main():
    scores = parse(RAW)
    print(f"Students: {len(scores)}")
    print(f"Average: {average(scores):.1f}")
    print(f"Top: {best(scores)}")


if __name__ == "__main__":
    main()

Students: 4
Average: 74.5
Top: cleo

The output is identical. What changed is that summary.py can now be reused, tested or imported elsewhere without reading scores or printing anything. Three files in one folder work because the script’s folder is on sys.path; once the number of files grows, you will want folders, which is the next lesson.

If the terminal and working directories in this lesson are unfamiliar, the fundamentals lesson on tools covers them.

print("loud loaded")


def shout(text):
    return text.upper() + "!"


if __name__ == "__main__":
    print(shout("direct"))

import loud
import loud

print(loud.shout("imported"))

$ python predict_run.py
loud loaded
IMPORTED!
$ python loud.py
loud loaded
DIRECT!

import math


def area(radius):
    return math.pi * radius**2


print(round(area(2), 2))

import math


def area(radius):
    return math.pi * radius**2


if __name__ == "__main__":
    print(round(area(2), 2))

Summary

• The first import of a module finds the file, runs it once, caches it in sys.modules and binds a name. Later imports reuse the cached module.
• if __name__ == "__main__": separates what a file does when run from what it offers when imported.
• sys.path decides which file an import finds, and its first entry depends on how you launched Python. 3.11+Requires version 3.11 or later -P removes that first entry.