Generators

Writing an iterator by hand, as in the last lesson, means an object, a next method, a counter in a closure and the { value, done } bookkeeping. A generator does the bookkeeping for you. It is a function that can pause in the middle, hand a value to its caller, and continue later from the same spot with all its local variables intact. The result is an iterator that you write like an ordinary loop.

function* and yield

A generator function is declared with function*. Calling it does not run the body. It returns a generator object, and the body runs a piece at a time as next() is called, up to the next yield.

function* steps() {
  console.log("start");
  yield 1;
  console.log("middle");
  yield 2;
  console.log("end");
  return 3;
}

const gen = steps();
console.log("created");
console.log(gen.next());
console.log(gen.next());
console.log(gen.next());
console.log(gen.next());

created
start
{ value: 1, done: false }
middle
{ value: 2, done: false }
end
{ value: 3, done: true }
{ value: undefined, done: true }

Follow the output line by line:

• created comes first and start has not been printed: calling steps() ran nothing.
• The first next() runs the body up to yield 1, which pauses there. The result is { value: 1, done: false }.
• The second next() resumes right after the yield, prints middle and stops at yield 2.
• The third next() prints end and reaches return 3. Now done is true, and the returned 3 is the final value.
• After that the generator is finished and every call gives { value: undefined, done: true }.

The return value is the one thing to remember: consumers such as for...of and spread ignore the final value, since they stop when done is true. Use yield for the values you want delivered.

Generators are iterators and iterables

The generator object has next(), and its [Symbol.iterator]() returns itself, so it works wherever the protocol does, and it is one-shot like any iterator.

function* letters() {
  yield "a";
  yield "b";
  return "ignored";
}

const gen = letters();
console.log(gen[Symbol.iterator]() === gen);
console.log([...gen], [...gen]);
console.log([...letters()], Array.from(letters()));

true
[ 'a', 'b' ] []
[ 'a', 'b' ] [ 'a', 'b' ]

The first spread uses up the generator, so the second is empty. Calling letters() again gives a new generator, and a fresh run. The return "ignored" value never appears.

Generators are also the easy way to make a class iterable. A generator method is written *name() {}, and with the computed name from lesson 7 it becomes *[Symbol.iterator]() {}:

class Countdown {
  constructor(from) {
    this.from = from;
  }

  *[Symbol.iterator]() {
    for (let n = this.from; n >= 1; n--) {
      yield n;
    }
  }
}

console.log([...new Countdown(3)]);

[ 3, 2, 1 ]

Compare it with the Countdown class of the previous lesson. The loop is the same one you would write to print the numbers, with yield in place of console.log. The state is just the loop variable, and since every call creates a new generator, each use starts afresh.

Infinite sequences

A generator pauses at each yield, so an infinite loop inside one is safe. It only runs as far as the consumer asks.

function* ids(prefix) {
  let n = 1;
  while (true) {
    yield `${prefix}-${n}`;
    n += 1;
  }
}

const orderIds = ids("order");
console.log(orderIds.next().value, orderIds.next().value);

order-1 order-2

for (const id of orderIds) {
  console.log(id);
  if (id.endsWith("4")) {
    break;
  }
}

order-3
order-4

console.log(orderIds.next());

{ value: undefined, done: true }

The generator keeps n between calls, so two next() calls give the first two ids, and the for...of loop continues with the third. The break ends the loop at order-4 and, as lesson 8 described, calls the iterator’s return(). For a generator that finishes it. The last line shows it: the generator is done, and a new next() gives nothing.

Delegating with yield*

yield* hands control to another iterable and yields each of its values in turn. It accepts any iterable: another generator, an array, a string.

function* inner() {
  yield "a";
  yield "b";
  return "inner finished";
}

function* outer() {
  yield 1;
  const result = yield* inner();
  yield result;
  yield* [2, 3];
  yield* "xy";
}

console.log([...outer()]);

[ 1, 'a', 'b', 'inner finished', 2, 3, 'x', 'y' ]

yield* inner() produced "a" and "b". The whole yield* expression evaluates to the generator’s return value, so result holds "inner finished", which the next line yields. Arrays and strings are delegated to item by item. Delegation makes recursion over nested data short, since a generator can yield* itself:

function* flatten(items) {
  for (const item of items) {
    if (Array.isArray(item)) {
      yield* flatten(item);
    } else {
      yield item;
    }
  }
}

const nested = [1, [2, [3, 4]], 5, [[6]]];
console.log([...flatten(nested)]);

[ 1, 2, 3, 4, 5, 6 ]

for (const value of flatten(nested)) {
  if (value > 2) {
    console.log("first above 2:", value);
    break;
  }
}

first above 2: 3

This is the generator form of the nested-array recursion from lesson 21. The loop at the bottom finds the first value above 2 and stops, and the rest of the array is never visited, because nothing asked for it.

Sending values in

next can take an argument. It becomes the value of the yield expression that the generator is paused on, which lets the caller send data in as well as read data out.

function* runningTotal() {
  let total = 0;
  while (true) {
    const amount = yield total;
    total += amount;
  }
}

const gen = runningTotal();
console.log(gen.next());
console.log(gen.next(5));
console.log(gen.next(10));
console.log(gen.next(-3).value);

{ value: 0, done: false }
{ value: 5, done: false }
{ value: 15, done: false }
12

The first next() has nothing to answer, so it only runs the generator up to its first yield, and any argument to that first call is discarded. After that, gen.next(5) makes yield total evaluate to 5, so amount is 5, the total is 5 and the loop pauses on the next yield with that total. This two-way style is used sparingly in everyday code, but it explains how the next two methods work.

return() and throw()

A generator object has two more methods besides next. return(value) ends the generator early, as if a return had been executed at the paused yield. throw(error) throws an error at the paused yield, where a try/catch inside the generator can handle it.

function* resilient() {
  while (true) {
    try {
      const input = yield "ready";
      console.log("got", input);
    } catch (error) {
      console.log("caught", error.message);
      yield "recovered";
    }
  }
}

const gen = resilient();
console.log(gen.next());
console.log(gen.next("a"));
console.log(gen.throw(new Error("boom")));
console.log(gen.next());
console.log(gen.return("bye"));
console.log(gen.next());

{ value: 'ready', done: false }
got a
{ value: 'ready', done: false }
caught boom
{ value: 'recovered', done: false }
{ value: 'ready', done: false }
{ value: 'bye', done: true }
{ value: undefined, done: true }

• next("a") resumed the paused yield "ready" with the value "a", so got a was printed.
• throw(new Error("boom")) raised the error at that same spot. The catch printed caught boom and the generator yielded "recovered".
• return("bye") ended it with { value: "bye", done: true }, and later calls report the end.

The most important use of return() is cleanup. A finally block in a generator runs when the generator ends early, and break in a loop triggers that.

function* reader() {
  try {
    yield 1;
    yield 2;
    yield 3;
  } finally {
    console.log("closed");
  }
}

for (const n of reader()) {
  console.log("read", n);
  if (n === 2) {
    break;
  }
}

read 1
read 2
closed

const [first] = reader();
console.log("first:", first);

closed
first: 1

Both the break and the destructuring that took one item stopped before the end, and both ran finally, which printed closed. If the generator holds a file or connection, this is where it is released.

Lazy pipelines

Small generators that take an iterable and return a generator compose like the array methods you know, but they process one item at a time and never build a full intermediate array.

function* naturals() {
  let n = 1;
  while (true) {
    yield n++;
  }
}

function* filter(iterable, test) {
  for (const item of iterable) {
    if (test(item)) {
      yield item;
    }
  }
}

map and take follow the same pattern:

function* map(iterable, fn) {
  for (const item of iterable) {
    yield fn(item);
  }
}

function* take(iterable, count) {
  if (count <= 0) {
    return;
  }
  for (const item of iterable) {
    yield item;
    count -= 1;
    if (count === 0) {
      return;
    }
  }
}

Connect the stages. Nothing runs until take asks for values:

const odd = filter(naturals(), (n) => n % 2 === 1);
const squares = map(odd, (n) => n * n);
console.log([...take(squares, 4)]);

[ 1, 9, 25, 49 ]

naturals() is infinite, yet the pipeline finishes: take asks for four items and then stops. Each number travels all the way through filter, map and take before the next one starts. Writing helpers like these by hand shows how the machinery works. The next lesson covers the built-in versions, which you will usually reach for.

Common pitfalls

• Expecting the body to run on the call. steps() only creates the generator. Nothing happens until the first next().
• Using return for values you want delivered. Consumers ignore it. Use yield.
• Forgetting that a generator is one-shot. Iterate the same generator object twice and the second pass is empty. Call the function again.
• Spreading an infinite generator. It never ends. Use take or break.
• Writing a generator as an arrow function. There is no such thing. Use function*, or a *method() in a class or object literal.
• Calling yield inside a callback. yield works only directly in the generator body, not in a nested function such as the callback of forEach. Use a for...of loop.
• Forgetting the star in yield*. yield inner() gives the generator object itself instead of its values.

Exercises

function* fibonacci() {
  let [a, b] = [0, 1];
  while (true) {
    yield a;
    [a, b] = [b, a + b];
  }
}

const first = [];
for (const value of fibonacci()) {
  if (first.length === 10) {
    break;
  }
  first.push(value);
}
console.log(first.join(" "));

0 1 1 2 3 5 8 13 21 34

const sequence = fibonacci();
sequence.next();
sequence.next();
console.log(sequence.next().value, sequence.next().value);

1 2

function node(value, left = null, right = null) {
  return { value, left, right };
}

function* inOrder(tree) {
  if (tree === null) {
    return;
  }
  yield* inOrder(tree.left);
  yield tree.value;
  yield* inOrder(tree.right);
}

const tree = node(4, node(2, node(1), node(3)), node(6, node(5), node(7)));
console.log([...inOrder(tree)].join(" "));
console.log(Math.max(...inOrder(tree)));
console.log(Array.from(inOrder(node(9))));
console.log([...inOrder(null)]);

1 2 3 4 5 6 7
7
[ 9 ]
[]

function* chunked(items, size) {
  for (let start = 0; start < items.length; start += size) {
    yield items.slice(start, start + size);
  }
}

console.log([...chunked([1, 2, 3, 4, 5], 2)]);
console.log([...chunked("abcdef", 3)]);
console.log([...chunked([], 4)]);

const pages = chunked([10, 20, 30], 2);
console.log(pages.next().value, pages.next().value, pages.next().done);

[ [ 1, 2 ], [ 3, 4 ], [ 5 ] ]
[ 'abc', 'def' ]
[]
[ 10, 20 ] [ 30 ] true

Summary

• A function* returns a generator object without running the body. Each next() runs to the next yield, which pauses with the yielded value. The return value, if any, ends the sequence and is ignored by loops and spread.
• A generator is an iterator and an iterable (one-shot). A *[Symbol.iterator]() method is the shortest way to make a class iterable.
• Infinite loops are safe in generators because they run only on demand.
• yield* delegates to any iterable, evaluates to the delegate’s return value, and makes recursion over trees and nested arrays concise.
• next(value) sends a value into a paused yield. return(value) ends the generator, and throw(error) raises an error at the yield. finally blocks run on early exit, including break.
• Small generators compose into lazy pipelines that never build the full intermediate list.

Next: iterator helpers, the built-in map, filter and take for iterators.