I have been using this concept in Javascript's redux-sagas & Python's asyncio but have failed to understand their underlying working. Who can explain to me how coroutines actually work?
I have been using this concept in Javascript's redux-sagas & Python's asyncio but have failed to understand their underlying working. Who can explain to me how coroutines actually work?
For further actions, you may consider blocking this person and/or reporting abuse
Adrian wrote an answer for a literal 5 year old, but from the OP's response it seems like he was expecting more. I'll go more into details, explaining the actual difference between the terms, while trying to keep it simple enough for a 5 year old to understand - assuming said 5 year old is a computer programmer. Gotta start that early to hit those job requirements...
Async
I think the term "async" is a bit misleading - the asynchronous thing is the underlying mechanism, and all these async frameworks are trying to give you a synchronous API to it.
The problem async trying to solve is that IOs are slow, but suffer very little performance hit when parallelized. So if you have 10 IOs that take one second each, running them synchronously one after the other will take 10 seconds:
But - if you can perform all of them in parallel it will take one second total:
The
for
loop is immediate - all it does is start the IOs(doesn't block) and add them to a data structure. In the second loop, the first iteration will take one second onpoll_one_that_is_ready()
. But during that second, the other 9 IOs were also running so they are also ready - and in the following iterationspoll_one_that_is_ready()
will be immediate. Since everything else will also be so much faster than 1 second that it can be considered immediate - the entire thing will run in 1 second.So, this is what async means - you start an IO and instead of waiting for it to finish you go to do other things(like sending more IOs).
Reactors
The problem with the above snippet is that writing code like this tends to get complex - especially when you have different kinds of IOs you need to send, with various dependencies on the results of previous IOs. That's the job of reactors - to hold many tasks, know which task waits on what, and decide which task to run next.
My
pending
container andwhile
loop form a super-simplified reactor - real asnyc frameworks will usually have a more complex reactors, that can handle multi-stage tasks and different kinds of IOs - but this is outside the scope of this answer. Another shortcoming of my "reactor" is that it can only deal with one task(with different arguments) - running "process_result" on the result. In reality you'll have different kind of IOs that should be handled differently - which brings us to the next section:Callback based async
Popularized by AJAX and Node.js, callbacks are a simple way to let the reactor know how to handle your IO. The idea is together with your IO to register a function(the "callback"), and once your IO is ready it'll call your function with it's result:
The reactor always receives the callback from
pending
, so it knows how to deal with the different kinds of IOs we register.This code is easier to manage than directly using the asnyc IO mechanism - but it's still has syntactic overhead compared to synchronous code. To make it look closer to simple synchronous code - we'll need:
Coroutines
Let's step back a bit from async to talk about coroutines. A regular function/procedure("routine") has a simple flow - you call it, it does it's job, and it returns. Let's take this function that gives us (a prefix of) the Fibonacci sequence:
(please ignore that for
limit < 2
you'll get 2 elements)This is a regular function - it does it's job and then returns. A coroutine, on the other hand, has many points during it's run where it can returns something to it's caller:
When we call
fib()
, instead of getting a list we'll get an iteratorit
. Each time we callnext(it)
the function will run until the nextyield
and return that value. So the first twonext(it)
will return1
, the third(theyield
in the first iteartion of thewhile True:
loop) will return2
, the fourth(second iteration) will return3
, the firth will return5
and so on. The loop is infinite - but it's controlled from the outside(by callingnext
) so the program doesn't get stuck.This was originally used for iterators, but it can also be used for:
Async with coroutines(finally!)
Instead of registering a callback function, our "callback" will be the rest of the coroutine after the
yield
:Now our
single_io
looks almost like synchronous code - and you just need some syntactic sugar to make it look like this:Yes, this was exactly the reply I was looking for! All very clear, and I def. got a better idea of how they work now in combination with async ops.
Within the last but one code snippet, I was wonderingโฆ
Why is there this call to
next(coroutine)
in thefor io in ios
loop?Because Python generators don't do anything when you call the function. They only execute code when you call
next
(orsend
) on them (or when you do something that calls theirnext
- like iterating on them with afor
loop or making alist
out of them)Async, concurrecy, coroutines or goroutines, from 10.000 feets in the air they are synonyms, the devil is in the detail but you haven't reached that stage I presume.
This is concurrency/async, you will finish all 3 tasks, in the same time span (this evening), by running a bit of each for a few minutes and then switch. From an outsider perspective (the User/programmer), you do all the 3 tasks in paralell.
The CPU does the switching so fast that it seems paralel, but is not. If your attention is disturbed, and instead of that 3minutes you stay 10min on a specific tasks, the entire thread is blocked. The best example in JS is if you block the main thread with a heavy operation, the page will freeze from the user points of view (because the user actions triggered events which are not handled because the thread is busy).
You can have multiple threads/tasks, but if you only have 1 Core, there is only 1 processator, so all of them will wait in line to be processed.
Now, paralelism ... is when you call your friend, with his own tablet , and put him to do your homework, and you remain with only 2 tasks (play and watch). Now you have 2 paralel tasks (threads) on 2 CPU cores, working side by side.
I made a collection of more advanced JavaScript topics, one of them is concurrency and async, I recommend check them out (especially the Aridanm and Event loop videos).
I think your explanation is missing the main appeal of async - that you don't need to block on IO.
To build on your story, I'd put it something like this:
The idea is that you don't just switch the tasks really fast to make it look like you are doing everything at once. You utilize the time you are waiting for something to happen(IO) to do other things that do require your direct attention.
Thank you, this was clear enough to grasp it a bit better. I thought coroutines were a specific pattern for async operations though, but from above comments, it seems like they're roughly the same thing.
No - your were right at first. Coroutines are one way to do async - there are other ways. There are also other uses for coroutines. I've written a detailed reply to the main post.
This. This makes so much sense. Really great explanation.
True, your completion explain how they should be used, in a correct way.
So couroutine is code based context switching, where you write the code and you write the code for context switching (where a function will stop and pass the execution to the caller).