Lambdas and higher-order functions

Sorting a list of names is easy. Sorting a list of records by team, then by age descending, then by name needs a way to tell sorted how to compare. In Python you do that by handing it a function. Functions are ordinary values, and this lesson shows what that makes possible.

Functions are values

A def creates a function object and binds it to a name, exactly like x = 5 binds a number. You can assign it to another name, put it in a list or dict, pass it as an argument, return it from another function, or check it with callable.

def shout(text):
    return text.upper() + "!"


speak = shout
print(speak("hello"))

HELLO!

Notice that speak = shout has no parentheses: it copies a reference to the function, it does not call it. A dict can turn a name chosen at run time into behaviour, a pattern that often replaces a long if/elif chain.

def shout(text):
    return text.upper() + "!"


def whisper(text):
    return text.lower() + "..."


styles = {"shout": shout, "whisper": whisper}
for name, style in styles.items():
    print(name, style("Hello"))

shout HELLO!
whisper hello...

A function that takes or returns another function is called a higher-order function. apply_twice takes one:

def shout(text):
    return text.upper() + "!"


def apply_twice(func, value):
    return func(func(value))


print(apply_twice(shout, "hi"))
print(apply_twice(lambda n: n * 2, 5))

HI!!
20

The second call passes a lambda, which the next section explains. make_multiplier returns a function: the inner multiply remembers factor through the closure you met in the scope lesson.

def make_multiplier(factor):
    def multiply(n):
        return n * factor

    return multiply


triple = make_multiplier(3)
print(triple(14))
print(callable(triple), callable(42))

42
True False

lambda

A lambda is a small anonymous function written as a single expression: lambda parameters: expression. The expression’s value is what the function returns. There is no return keyword and no room for statements.

words = ["banana", "kiwi", "Apple", "cherry"]

print(sorted(words))
print(sorted(words, key=len))
print(sorted(words, key=str.lower))
print(sorted(words, key=lambda w: w[-1]))

['Apple', 'banana', 'cherry', 'kiwi']
['kiwi', 'Apple', 'banana', 'cherry']
['Apple', 'banana', 'cherry', 'kiwi']
['banana', 'Apple', 'kiwi', 'cherry']

min and max take a key= too:

words = ["banana", "kiwi", "Apple", "cherry"]

print(max(words, key=len))
print(min(words, key=lambda w: w.lower()))

banana
Apple

A lambda is an ordinary function, but with no name of its own:

add = lambda a, b: a + b
print(add(2, 3))
print(add.__name__)


def add_def(a, b):
    return a + b


print(add_def.__name__)

5
<lambda>
add_def

The plain sorted(words) puts "Apple" first because uppercase letters sort before lowercase. The key= argument fixes that, and it is the most common home for a lambda. sorted, min, max and list.sort all call the key function once per item and compare the results instead of the items.

• key=len sorts by length. banana and cherry tie at six letters and keep their original order, because Python’s sort is stable.
• key=str.lower passes an existing method, with no lambda needed. Reach for a lambda only when no ready-made function does the job.
• key=lambda w: w[-1] sorts by the last letter.

The last lines show why assigning a lambda to a name is discouraged (PEP 8): the function’s name is the unhelpful <lambda>, which appears in tracebacks. If a function deserves a name, use def.

map, filter and comprehensions

map(func, items) applies a function to every item; filter(func, items) keeps the items for which the function returns a true value. Both return lazy iterators, so wrap them in list to see the results.

numbers = [1, 2, 3, 4, 5, 6]

print(list(map(lambda n: n * n, numbers)))
print(list(filter(lambda n: n % 2 == 0, numbers)))

[1, 4, 9, 16, 25, 36]
[2, 4, 6]

The same results with comprehensions:

numbers = [1, 2, 3, 4, 5, 6]

print([n * n for n in numbers])
print([n for n in numbers if n % 2 == 0])

[1, 4, 9, 16, 25, 36]
[2, 4, 6]

With an existing function, map needs no lambda:

print(list(map(str.upper, ["a", "b"])))
print(list(map(int, ["1", "2", "3"])))

['A', 'B']
[1, 2, 3]

And the catch with lazy iterators:

lazy = map(str, [1, 2, 3, 4])
print(next(lazy))
print(list(lazy))
print(list(lazy))

1
['2', '3', '4']
[]

The comprehensions do the same work as map and filter with a lambda, and most Python programmers find them easier to read. Prefer a comprehension when you would otherwise write a lambda inside map or filter. Use map when you already have a named function, as in map(int, strings).

The last example shows the catch with lazy iterators: after next(lazy) took the first item and list(lazy) took the rest, the iterator is used up, so a second list(lazy) is empty.

Sorting by several keys

Because key= may return a tuple, and tuples compare item by item, one lambda can express a multi-level sort. Negating a number reverses its order within the tuple.

people = [
    {"name": "Ada", "team": "math", "age": 36},
    {"name": "Grace", "team": "navy", "age": 85},
    {"name": "Alan", "team": "math", "age": 41},
    {"name": "Katherine", "team": "nasa", "age": 101},
    {"name": "Barbara", "team": "math", "age": 36},
]

# team A to Z, then oldest first, then name A to Z
ordered = sorted(people, key=lambda p: (p["team"], -p["age"], p["name"]))
for person in ordered:
    print(person["team"], person["age"], person["name"])

math 41 Alan
math 36 Ada
math 36 Barbara
nasa 101 Katherine
navy 85 Grace

The same result is possible with stable sorts, least important key first. operator.itemgetter("age") is a ready-made function equivalent to lambda p: p["age"].

from operator import itemgetter

people = [
    {"name": "Ada", "team": "math", "age": 36},
    {"name": "Grace", "team": "navy", "age": 85},
    {"name": "Alan", "team": "math", "age": 41},
    {"name": "Katherine", "team": "nasa", "age": 101},
    {"name": "Barbara", "team": "math", "age": 36},
]

second = sorted(people, key=itemgetter("name"))
second = sorted(second, key=itemgetter("age"), reverse=True)
second = sorted(second, key=itemgetter("team"))
for person in second:
    print(person["team"], person["age"], person["name"])

math 41 Alan
math 36 Ada
math 36 Barbara
nasa 101 Katherine
navy 85 Grace

The tuple trick works when you can negate the field (numbers). For text you cannot negate, use the second approach: sort several times, least important key first, and rely on stability so earlier orderings survive within ties.

Common pitfalls

• Late binding in lambdas. A lambda created in a loop looks up the loop variable when it is called, not when it is created. See the example below.
• Cramming logic into a lambda. If it needs a conditional chain or more than a short expression, write a def with a name and a docstring.
• Calling when you meant to pass. sorted(words, key=len()) fails; the key is the function len itself. Parentheses call it.
• Forgetting that map and filter are lazy. Printing one shows an object, not results. Wrap it in list(...), and remember it can be used only once.
• Assigning the result of list.sort(). items = items.sort() sets items to None, because list.sort() sorts in place and returns nothing. sorted(items) returns a new list.

funcs = [lambda: i for i in range(3)]
print([f() for f in funcs])

funcs = [lambda i=i: i for i in range(3)]
print([f() for f in funcs])

[2, 2, 2]
[0, 1, 2]

All three lambdas share one i, and by the time they run the loop has finished with i equal to 2. Giving the lambda a parameter with a default, i=i, captures the current value at creation time.

101 Katherine
85 Grace
41 Alan
36 Ada
36 Barbara
['Ada', 'Alan', 'Barbara']

people = [
    {"name": "Ada", "team": "math", "age": 36},
    {"name": "Grace", "team": "navy", "age": 85},
    {"name": "Alan", "team": "math", "age": 41},
    {"name": "Katherine", "team": "nasa", "age": 101},
    {"name": "Barbara", "team": "math", "age": 36},
]

by_age = sorted(people, key=lambda p: (-p["age"], p["name"]))
for person in by_age:
    print(person["age"], person["name"])

math_names = sorted(p["name"] for p in people if p["team"] == "math")
print(math_names)

101 Katherine
85 Grace
41 Alan
36 Ada
36 Barbara
['Ada', 'Alan', 'Barbara']

words = ["pear", "fig", "banana", "kiwi", "apple"]
print(sorted(words, key=lambda w: (len(w), w)))

['fig', 'kiwi', 'pear', 'apple', 'banana']

OPERATIONS = {
    "add": lambda a, b: a + b,
    "sub": lambda a, b: a - b,
    "mul": lambda a, b: a * b,
}


def calculate(op, a, b):
    func = OPERATIONS.get(op)
    if func is None:
        return None
    return func(a, b)


print(calculate("add", 2, 3))
print(calculate("mul", 4, 5))
print(calculate("pow", 2, 3))

5
20
None

Summary

• A function is a value: assign it, store it, pass it to or return it from other functions.
• lambda args: expression makes a small anonymous function. It shines as a key= and is a poor fit for anything with a name or several steps.
• key= can return a tuple for multi-level sorts, and a comprehension is usually clearer than map or filter with a lambda.

Next: recursion in Python, and the limits Python puts on it.