Memory

Time is not the only cost. A program that needs 6 GB on a 4 GB container gets killed, however fast it is. The fundamentals lesson on memory and references describes objects and references in general. This lesson measures them in CPython. Every number printed here was checked on 64-bit CPython 3.15. Sizes are implementation details, and they change between versions and builds.

sys.getsizeof is shallow

sys.getsizeof(obj) returns the size of one object’s own memory. For a container, that means the container and its array of references, not the objects they point to:

import sys

numbers = list(range(1_000))
print("list object:", sys.getsizeof(numbers))
print("one small int:", sys.getsizeof(5))


def deep_size(obj, seen=None):
    """Size of obj plus everything it contains, each object once."""
    seen = set() if seen is None else seen
    if id(obj) in seen:
        return 0
    seen.add(id(obj))
    size = sys.getsizeof(obj)
    if isinstance(obj, (list, tuple, set, frozenset)):
        size += sum(deep_size(item, seen) for item in obj)
    elif isinstance(obj, dict):
        size += sum(deep_size(k, seen) + deep_size(v, seen)
                    for k, v in obj.items())
    return size


print("list and its ints:", deep_size(numbers))
print("same int 1000 times:", deep_size([5] * 1_000))

list object: 8056
one small int: 28
list and its ints: 36056
same int 1000 times: 8084

The list is 56 bytes of header plus 8 bytes per reference. Each distinct int is another 28 bytes, so the ints cost more than the list itself. A list holding the same object 1,000 times pays for it once, which is why deep_size tracks ids. Even this deep size is an estimate: it misses instance attributes, memory allocator overhead and anything owned by C code. For the real answer, ask the allocator.

tracemalloc: what was actually allocated

tracemalloc hooks Python’s memory allocator and records every block, with the line that allocated it. get_traced_memory() returns the current and the peak traced size since start(). Peak is what decides whether a process fits in memory:

import tracemalloc


def peak_of(func):
    tracemalloc.start()
    result = func()
    _now, peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    return result, peak


def with_list():
    squares = [n * n for n in range(200_000)]   # all at once
    return sum(squares)


def with_generator():
    return sum(n * n for n in range(200_000))   # one at a time


a, list_peak = peak_of(with_list)
b, gen_peak = peak_of(with_generator)
print("same answer:", a == b)
print("generator peak is 100x smaller:", gen_peak * 100 < list_peak)

same answer: True
generator peak is 100x smaller: True

The list version holds 200,000 ints at once. The generator holds one at a time, so its peak does not grow with the input. Tracing makes allocation slower, often by a factor of two or more, so turn it on for a measurement, not in production. tracemalloc.reset_peak() starts a new peak without discarding the traces.

Which line grew?

Snapshots let you compare memory before and after a step, grouped by source line:

import tracemalloc

tracemalloc.start()
before = tracemalloc.take_snapshot()

cache = {}
for i in range(20_000):
    cache[i] = f"row-{i}"                  # the line that grows
labels = [str(i) for i in range(500)]

after = tracemalloc.take_snapshot()
tracemalloc.stop()

top = after.compare_to(before, "lineno")[0]
frame = top.traceback[0]
print("biggest growth at line", frame.lineno)
print("in this file:", frame.filename.endswith("snapshot_diff.py"))
print("new blocks at least 20000:", top.count_diff >= 20_000)

biggest growth at line 8
in this file: True
new blocks at least 20000: True

Hunting a leak in a running service

The same tool, used in order, finds most leaks in a process whose memory keeps growing:

1. Start the process with python -X tracemalloc=25, which traces from start-up and keeps 25 frames per allocation, so allocations in shared helpers show their call path.
2. Let it warm up (caches fill, imports finish), then take a snapshot.
3. Send it ordinary traffic for a while, and take a second snapshot.
4. Print the top ten entries of second.compare_to(first, "traceback"). Repeat steps 3 and 4: a line whose size_diff grows every time is the leak. One that grew once and then stopped is a cache that filled.
5. Find out what keeps those objects alive. gc.get_referrers(obj) lists the objects that refer to one of them. Memory management explains references, cycles and weak references, the usual culprits.

Steps 2 to 4 need a way to run code inside the service, such as a debug endpoint or a signal handler that writes each snapshot to a file with snapshot.dump(path). tracemalloc.Snapshot.load(path) reads them back in a separate process for the comparison. tracemalloc sees only memory requested through Python’s allocators, so memory that a C library gets straight from malloc is invisible to it. For that, use a native memory profiler such as the third-party memray.

__slots__: smaller objects

A normal instance keeps its attributes in a per-instance namespace. A class with __slots__ gets fixed storage for the named attributes instead, and no namespace at all:

import sys
import tracemalloc


class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y


class SlotPoint:
    __slots__ = ("x", "y")

    def __init__(self, x, y):
        self.x, self.y = x, y

def traced_bytes(cls, n=100_000):
    tracemalloc.start()
    points = [cls(0.0, 0.0) for _ in range(n)]
    used, _peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    return used


print("getsizeof:", sys.getsizeof(Point(0, 0)), sys.getsizeof(SlotPoint(0, 0)))
plain, slotted = traced_bytes(Point), traced_bytes(SlotPoint)
print("slots use less:", slotted < plain)
print("saving at least 30%:", slotted < 0.7 * plain)
print(f"measured saving: {1 - slotted / plain:.0%}", file=sys.stderr)

getsizeof: 48 48
slots use less: True
saving at least 30%: True

getsizeof reports the same 48 bytes for both, because since 3.12 CPython stores a plain instance’s attribute values in an area that getsizeof does not count (3.11 prints 56 for Point). tracemalloc sees the difference. The measured saving on standard error was 42% on my machine, on both the default and the free-threaded 3.15 build, so the 30% threshold has room. The saving is per instance, so it matters for hundreds of thousands of small objects and not at all for a dozen. The cost is flexibility. You cannot add attributes that are not listed, and multiple inheritance from two slotted bases with non-empty slots fails. @dataclass(slots=True) generates the slots for you (Data classes), and Class customisation shows how slots work underneath, as descriptors.

Zero-copy bytes with take_bytes

3.15+Requires version 3.15 or later bytearray.take_bytes(n=None) removes the first n bytes from a bytearray and returns them as bytes. With no argument it takes everything and leaves the bytearray empty:

buffer = bytearray(b"GET /index\nHost: example\npartial")

line = buffer.take_bytes(buffer.find(b"\n") + 1)   # first n bytes
print(line, type(line).__name__)
print(buffer)

rest = buffer.take_bytes()                         # everything
print(rest)
print("buffer now empty:", len(buffer) == 0)

b'GET /index\n' bytes
bytearray(b'Host: example\npartial')
b'Host: example\npartial'
buffer now empty: True

The point is the copy it avoids. The old idiom bytes(buffer) copies the whole buffer, so for a moment you hold two. In CPython, taking all the bytes hands the existing memory over to the new bytes object:

import io
import tracemalloc

SIZE = 8_000_000
SOURCE = io.BytesIO(b"x" * SIZE)


def peak_of(finish):
    buffer = bytearray(SIZE)               # exact size, filled in place
    SOURCE.seek(0)
    SOURCE.readinto(buffer)
    tracemalloc.start()
    data = finish(buffer)
    _now, peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    return data, peak


copied, copy_peak = peak_of(bytes)
taken, take_peak = peak_of(bytearray.take_bytes)
print("same bytes:", copied == taken)
print("bytes() allocated a second copy:", copy_peak >= SIZE)
print("take_bytes() allocated almost nothing:", take_peak < SIZE // 100)

same bytes: True
bytes() allocated a second copy: True
take_bytes() allocated almost nothing: True

Two caveats. Taking only part of the buffer still copies the bytes that remain. And a bytearray grown with += usually has spare capacity: in my measurements on 3.15.0, taking all of such a buffer still allocated a block of the full size. Fill a buffer of the right size with readinto when you can, and measure your own case.

When not to optimise memory

If the process fits comfortably in its memory limit, memory work buys nothing. Reach for it when a process is killed, when it swaps, when you run many copies of it, or when peak usage grows with input size and the input will grow. __slots__ on a class with ten instances, or a generator where a list of 50 items is clearer, is noise. Streaming has a real cost too: a generator can be consumed only once, and you cannot len() it or index it.

Common pitfalls

Summing getsizeof and calling it the total. It is shallow, and it misses allocator overhead and inline attribute storage.

Measuring the current size instead of the peak. A function can return a small result after briefly using gigabytes.

Leaving tracemalloc on. It slows allocation and itself uses memory.

Holding a reference you forgot. A cache, a global list or a closure keeps objects alive. Snapshots point at the line that allocated them, and gc.get_referrers shows what holds them.

import tracemalloc

LINES = [f"{i},sensor-{i % 50},{i % 97}.5" for i in range(50_000)]


def load_dicts(lines):
    rows = []
    for line in lines:
        rid, sensor, value = line.split(",")
        rows.append({"id": int(rid), "sensor": sensor,
                     "value": float(value)})
    return sum(r["value"] for r in rows)


def peak_of(loader):
    tracemalloc.start()
    total = loader(LINES)
    _now, peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    return total, peak

def load_stream(lines):
    total = 0.0
    for line in lines:
        total += float(line.rsplit(",", 1)[1])
    return total


before_total, before = peak_of(load_dicts)
after_total, after = peak_of(load_stream)
print("same total:", before_total == after_total)
print("peak at most half:", after * 2 <= before)

same total: True
peak at most half: True

import sys

flat = [0] * 1_000
nested = [flat]
copies = [flat] * 1_000

print(sys.getsizeof(flat))
print(sys.getsizeof(nested))
print(sys.getsizeof(copies))

8056
64
8056

def split_lines(buffer):
    """Remove and return every complete line in buffer."""
    lines = []
    while (n := buffer.find(b"\n")) != -1:
        lines.append(bytes(buffer[:n + 1]))   # copy the slice ...
        del buffer[:n + 1]                     # ... then shift the rest
    return lines


buffer = bytearray(b"alpha\nbeta\ngam")
print(split_lines(buffer), buffer)
buffer += b"ma\n"
print(split_lines(buffer), buffer)

[b'alpha\n', b'beta\n'] bytearray(b'gam')
[b'gamma\n'] bytearray(b'')

def split_lines(buffer):
    """Remove and return every complete line in buffer."""
    lines = []
    while (n := buffer.find(b"\n")) != -1:
        lines.append(buffer.take_bytes(n + 1))
    return lines


buffer = bytearray(b"alpha\nbeta\ngam")
print(split_lines(buffer), buffer)
buffer += b"ma\n"
print(split_lines(buffer), buffer)

[b'alpha\n', b'beta\n'] bytearray(b'gam')
[b'gamma\n'] bytearray(b'')

Summary

• sys.getsizeof is shallow and misses inline attribute storage. Use tracemalloc for what was really allocated.
• Peak memory decides whether a process fits. get_traced_memory() gives it, and snapshots with compare_to show the line that grew.
• __slots__ cuts per-instance memory, and streaming cuts peak memory. Both have costs in flexibility.
• 3.15+Requires version 3.15 or later bytearray.take_bytes() turns a buffer into bytes without copying when you take all of it.
• Next: Optimising Python code.