From the last article, we have talked about how generators could be considered as coroutines, but cannot be fully accepted as coroutines, and since PEP 492 we have finally native coroutine objects in Python.
- constructed with
- basically built on generator without such iterator features(no
__iter__()), but sharing the interface with generator(such as
- equipped with
__await__()such that it is usable with
__await__()returns an iterator)
To see what exactly those things mean, we’ll observe a few simple examples that demonstrate the behaviors of native coroutines.
But let’s first look into how the structure of coroutine looks like.
In fact, they are not brand new objects, but has an object structure very similar with that of generators, where the core method
send() is shared. So we can expect that the actual logic of a coroutine executed under the hood are somewhat similar to that of a generator, having suspended execution states at some points.
This expectation is in a sense true, although it doesn’t have to be - a native coroutine doesn’t need to suspend at all, but it could. This is largely because native coroutines provide the programmers freedom to choose what to do with them - as long as the objects following the
await expression are either native coroutines or objects that return iterators through
Thus, a coroutine is actually a set of chained coroutines, such that at the bottom level of the chain those awaited objects must return non-coroutine iterators. So it is up to the programmer to return generators as those iterators(well, this is why we use native coroutines), but it doesn’t have to(she or he can just return non-generator iterators).
Now let’s observe how native coroutines work using an actual native coroutine object. But let’s briefly discuss
yield from, which is related to
While reading PEP 492, you must have come across the syntax
yield from <subgenerator>. I haven’t had time to discuss this keyword, but it is simply handing in the current control flow to
<subgenerator>, so that we can have a chain of generators. The reason why I introduce this syntax is because it is essential to call another coroutine inside a coroutine using
await, which - according to PEP 492 - uses the same implementation as
So we have
close() for a native coroutine object as well as a generator. Since the docs keep saying “delegating” those methods to the iterator returned from
__await__(), I guess looking into only
send method will be sufficient for our purpose.
Let’s augment a little bit of our example(a part of code is from https://stackoverflow.com/a/60118660):
async def await1() -> str: print("await 1!!!") return "foo" class Await2: def __await__(self) -> str: print("before executing Await2") val = yield "await from Await2!!!" print("value received from send(): ", val) val2 = yield "await from Await2!!! - 2" print("value received from send(): ", val2) return "actual return value" async def example() -> int: print("coroutine runs!") print(await await1()) print("await 1 ended") print(await Await2()) print("Await 2 ended") print("end!") return 1 if __name__ == "__main__": coro = example() print("---send None---") coro.send(None) print("---send 1---") coro.send(1) print("---send 2---") coro.send(2)
which will give this result:
---send None--- coroutine runs! await 1!!! foo await 1 ended before executing Await2 ---send 1--- value received from send(): 1 ---send 2--- value received from send(): 2 actual return value Await 2 ended end! Traceback (most recent call last): # omitted
The result is in line with our previous discussion about the internal structure of native coroutines. If a coroutine doesn’t suspend with
await1()), the execution continues up until it hits a
”before executing Await2”). Whereas generators halt at the next
yield, native coroutines traverse their chained sub-coroutines(yes,
await is not
yield - it is more like
yield from, allowing a chained sequence of coroutines). After coming across
Await2, the result is almost similar to that of generators. One difference is that we can’t see those yielded values(
”await from Await2!!!”) outside
__await__(), but only the returning value(
"actual return value”).
So here we can confirm that the coroutine object doesn’t do more than what is specified in the documentation, and it is the programmer who implement the details. Perhaps that is why most of the references(including the ones I recommended in the preface of this series of articles) don’t mention much about these methods of the coroutine object - there is nothing much to talk about!
So our takeaways from the discussion so far are:
- native coroutines are similar in structure with generators: instead of
yield fromthey have
awaitfor chaining executions
await, not only coroutine for chaining, you can use any
- at the bottom of a chaining of coroutines, we have iterators such as generators with
Therefore you can understand a native coroutine object in Python - under the context of asynchronous programming - as an object with execution information that is chainable with other coroutines and other awaitable objects, to which it yields the current control flow and waits until those awaitable objects returns values, just like generator objects do with
You might have thought why most of the discussions about coroutines and asyncio out there don’t explain much about the native coroutine object but suddenly move to the asyncio library(or any other similar libraries like trio or curio). Now that makes sense - a native coroutine is simply a frame or an interface, and those libraries fill in the actual behaviors.
However, it should be daunting for us to investigate even a single library in details. Hence, in the next article, we will pick one of them to briefly sketch how asynchronous workflows could be implemented with several relevant and important concepts involved.