Generators: Advanced Iteration and More

Presenter: Erik Swanson

Generators and iterators allow users to define their own control flow. Since their introduction in Python 2.2, generators have grown to allow sending and receiving values, as well as delegating to sub-generators, making generators a form of coroutines.

In this talk, we’ll start by introducing the basics of generators. We’ll explain what’s going on under the hood, how they relate to python’s iterators, and how they suspend computation. We’ll see some examples of iteration using generators, as well as how generators can be used to simplify the implementation of context managers and other code.

In the second part of this talk, we’ll cover two more complicated use cases of generators:

1. We’ll simplify a complex data pipeline using generators, increasing memory efficiency in the process. This section will cover how generators can be used both to send and receive data.

2. We’ll create a concurrent event-loop built from generator-based coroutines. In this section, we’ll explain delegating to sub-generators using “yield from”.

This talk is designed to be followed along interactively, so if possible, come with a computer that has Python >= 3.5 installed (and optionally, Jupyter Notebooks installed)