Lazy imports (Python) | Programming Tutorials

Every import at the top of a file runs that module, and the modules it imports, before your program does anything. A command-line tool that imports a dozen large libraries can take a noticeable fraction of a second to print its --help, even though a typical run uses two of them. Requires version 3.15 or later Python 3.15 adds explicit lazy imports (PEP 810): a way to say “bind this name now, but do the work of importing when I first use it.”

Everything in this lesson needs Python 3.15 or later, with one exception that is labelled. On 3.14 and earlier, lazy import is a SyntaxError.

The lazy keyword

Put lazy in front of an import statement. It works with both forms:

import sys
import types

lazy import statistics

print("statistics" in sys.modules)
print(type(globals()["statistics"]) is types.LazyImportType)

Output

False
True

The program has passed the import line, yet statistics is not in sys.modules. The name statistics exists, but it is bound to a lightweight placeholder of type types.LazyImportType. (Reading the name through globals() lets us look at the placeholder without triggering it. An ordinary use of the name triggers it, as the next step shows.)

Using the name is what loads the module:

print(statistics.mean([2, 4, 9]))

print("statistics" in sys.modules)
print(type(globals()["statistics"]) is types.LazyImportType)

Output

5
True
False

On first use Python imports the real module and replaces the placeholder, so later uses cost nothing extra. From then on the name behaves exactly like a normal import.

lazy from pathlib import Path works the same way: Path is bound when the first reference to it happens. The convention that imports live together at the top of the file is preserved, which is the whole appeal. The alternative used until now was to move an import inside the one function that needs it, scattering imports through the file.

Seeing the savings

Here is a small command-line program that only needs statistics when given --stats. First the eager version, with ordinary imports. The loaded() function reports which of three heavy modules are present:

import asyncio
import statistics
import sys
import zipfile

HEAVY = ["asyncio", "statistics", "zipfile"]


def loaded():
    return [name for name in HEAVY if name in sys.modules]


def main(args):
    if args[:1] == ["--stats"]:
        print("mean:", statistics.mean(float(x) for x in args[1:]))
    else:
        print("usage: cli --stats NUMBER...")
    print("loaded:", loaded())


main(["--help"])
main(["--stats", "1", "2", "6"])

Output

usage: cli --stats NUMBER...
loaded: ['asyncio', 'statistics', 'zipfile']
mean: 3.0
loaded: ['asyncio', 'statistics', 'zipfile']

All three were loaded even for the --help run, which used none of them. Now the same program with the three imports marked lazy, and nothing else changed (the sys import stays ordinary, because the program needs it immediately):

import sys

lazy import asyncio
lazy import statistics
lazy import zipfile

HEAVY = ["asyncio", "statistics", "zipfile"]


def loaded():
    return [name for name in HEAVY if name in sys.modules]


def main(args):
    if args[:1] == ["--stats"]:
        print("mean:", statistics.mean(float(x) for x in args[1:]))
    else:
        print("usage: cli --stats NUMBER...")
    print("loaded:", loaded())


main(["--help"])
main(["--stats", "1", "2", "6"])

Output

usage: cli --stats NUMBER...
loaded: []
mean: 3.0
loaded: ['statistics']

The help run loaded nothing. The --stats run loaded only statistics, the one module it used. asyncio and zipfile were never imported.

To put numbers on it, ask Python to time every import with -X importtime. It prints one line per module to standard error, with the time in microseconds, so redirect or filter it:

$ python -X importtime cli_eager.py 2>&1 | grep -E " (asyncio|statistics|zipfile)$"
import time:       400 |      47289 | asyncio
import time:       743 |       6854 | statistics
import time:       917 |       5540 | zipfile

$ python -X importtime cli_lazy.py 2>&1 | grep -E " (asyncio|statistics|zipfile)$"
import time:      1308 |      16439 | statistics

The first column is the module’s own time and the second is cumulative, including everything it imported in turn. Your numbers will differ from these, and will vary from run to run, so compare orders of magnitude and repeat the measurement. In the lazy run only statistics appears, and only because the second call used it.

Where lazy is not allowed

The point of laziness is that the import happens at some unknown later moment, which is only safe where Python can treat it as a module-level name. So the rules are strict:

lazy import is allowed only at module scope. Inside a function, a class body or a try block, it is a SyntaxError.
lazy from module import * is a SyntaxError, and so is a lazy __future__ import.

def f():
    lazy import json
    return json

Output

  File "in_function.py", line 2
    lazy import json
    ^^^^^^^^^^^^^^^^
SyntaxError: lazy import not allowed inside functions

Two behaviours deserve care. First, errors move. A normal import of a missing module fails when the program starts. A lazy one fails at the first use, possibly deep inside a run. The traceback shows both the original lazy import line and the use, so it can be traced:

lazy import no_such_module_anywhere

print("start")
no_such_module_anywhere.run()

Output

start
Traceback (most recent call last):
  File "late_failure.py", line 1, in <module>
    lazy import no_such_module_anywhere
ImportError: lazy import of 'no_such_module_anywhere' raised an exception during resolution

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "late_failure.py", line 4, in <module>
    no_such_module_anywhere.run()
    ^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'no_such_module_anywhere'

Second, a module’s own top-level code runs later too. If importing a module registers a plugin or prints a banner, that now happens at first use. Do not make a module lazy when something else relies on its side effects at start-up.

Older Pythons and global switches

A file that contains lazy import cannot even be parsed by 3.14, so a project that supports both needs another route. A module can list names in __lazy_modules__, and ordinary import statements for those modules become lazy on 3.15. On older versions the list is just an unused variable, and the imports are eager but correct. Requires version 3.15 or later

import sys

__lazy_modules__ = ["statistics"]

import os
import statistics

print("os loaded:", "os" in sys.modules)
print(statistics.mean([2, 4]))
print("statistics loaded:", "statistics" in sys.modules)

Output

os loaded: True
3
statistics loaded: True

The output is the same on every version, because the last line comes after the first use. What differs is whether statistics was loaded before that point. The file parses and runs on older versions too; it was checked here on 3.13 as well as 3.15.

There are also global controls, for experimenting without editing source. python -X lazy_imports=all (or the environment variable PYTHON_LAZY_IMPORTS) makes every import lazy where the rules allow it, and normal, the default, honours only the lazy keyword. The functions sys.set_lazy_imports() and sys.set_lazy_imports_filter() change that behaviour from inside a program. These are for application authors who want to try a whole program lazily. Libraries should not change them.

ExercisePredict the output

Without running it, say what this prints under Python 3.15. Then run it.

import sys

lazy import csv
lazy import json

print("json" in sys.modules)
json.dumps({})
print("json" in sys.modules, "csv" in sys.modules)

Show solution

False
True False

Neither module has loaded at the first print. json.dumps({}) uses json, which loads it. csv is never used, so it stays unloaded, and the second line reports True False.

ExerciseWhich lines are errors?

For each statement, say whether it is allowed in Python 3.15, assuming it is at the top of a file unless stated otherwise: lazy import os, lazy from os import path, lazy from os import *, lazy import json inside a function, import lazy.

Expected: five lines in the form statement: allowed or statement: SyntaxError.

Show solution

lazy import os: allowed.
lazy from os import path: allowed.
lazy from os import *: SyntaxError, since star imports cannot be lazy.
lazy import json inside a function: SyntaxError, since lazy imports are only allowed at module scope.
import lazy: allowed as syntax, though it would fail unless a module called lazy exists. lazy is a soft keyword, meaning it only acts as a keyword at the start of an import statement, so it remains usable as an ordinary name elsewhere.

ExerciseMeasure a CLI before and after

Save this script as startup.py. It imports two modules and uses neither:

import json
import statistics

print("ok")

Count how many lines of python -X importtime startup.py 2>&1 end in json or statistics, with grep -cE '\| +(json|statistics)$'. Then change both imports to lazy import, and count again.

Expected: the script prints ok both times. The count is 2 before the change and 0 after it.

Show solution

$ python -X importtime startup.py 2>&1 | grep -cE '\| +(json|statistics)$'
2

After changing the first two lines to lazy import json and lazy import statistics:

$ python -X importtime startup.py 2>&1 | grep -cE '\| +(json|statistics)$'
0

The modules are never used, so a lazy import never loads them, and their lines vanish from the report. The cumulative times in the report differ from run to run, which is why the exercise counts lines instead.

If a module you made lazy still appears at start-up, something else imports it eagerly, such as another module in your program or one of your dependencies. Lazy imports only defer your own import statements.

Summary

Requires version 3.15 or later lazy import x and lazy from x import y bind a placeholder and load the module on first use. They are allowed only at module scope.
python -X importtime shows what each import costs. Lazy imports help most in programs that import much more than any one run uses.
Errors and import side effects move to the first use. __lazy_modules__ gives older Pythons code that is still valid, although eager.