[00:00]
Today we're going to learn about
[00:01]
asynchronous programming in Python with
[00:03]
Async IO as quickly as possible. So let
[00:05]
us get right into it.
[00:11]
[Music]
[00:15]
>> All right, so as always when it comes to
[00:17]
these tutorials where I try to cover
[00:18]
something as quickly as possible, this
[00:20]
is not going to be a deep dive. We're
[00:22]
not going to go into a lot of details. I
[00:24]
wouldn't even call it a crash course.
[00:25]
It's more like me giving you a very very
[00:27]
quick introduction into the topic and
[00:29]
then you can continue to study it on
[00:31]
your own or if you want to you can leave
[00:33]
me a comment in the comment section down
[00:34]
below and let me know that you're
[00:36]
interested in a more detailed course so
[00:38]
maybe I can do that as well on my
[00:39]
channel but asynchronous programming
[00:42]
belongs to the category of concurrent
[00:44]
programming in Python and there are
[00:45]
three major ways to do concurrent
[00:47]
programming in Python one is
[00:49]
asynchronous programming another one is
[00:51]
multi-threading and another one is
[00:53]
multipprocessing for the last two I
[00:55]
already have two videos on my channel
[00:57]
similar to this one where I cover them
[00:59]
as quickly as possible. So you can take
[01:00]
a look at them if you want to. And today
[01:03]
we're going to talk about asynchronous
[01:04]
programming. Now in a nutshell, you want
[01:06]
to use multipprocessing when you have a
[01:08]
lot of CPUbound tasks that you want to
[01:10]
parallelize. So heavy computations that
[01:13]
you want to do simultaneously on
[01:15]
multiple CPU cores. You actually want to
[01:17]
have multiple processes. Uh you don't
[01:19]
want to be limited by the global
[01:20]
interpreter log. Multi-threading is a
[01:23]
little bit more I wouldn't necessarily
[01:25]
say exotic, but it's a bit odd in Python
[01:28]
because you have the global interpreter
[01:29]
lock and you don't have real
[01:30]
multi-threading unless you release the
[01:32]
global interpreter lock. But
[01:34]
essentially, you want to use it when you
[01:35]
don't want to manually handle the
[01:38]
control. So you don't want to manually
[01:39]
switch between the different threads and
[01:41]
you also maybe have to work with
[01:43]
something that doesn't support
[01:44]
asynchronous programming. But I want I
[01:46]
don't want to talk about this too much.
[01:47]
I want to focus on asynchronous
[01:49]
programming today. uh which is the topic
[01:51]
of this video. So let us go into our
[01:53]
coding directory. In my case, I'm going
[01:55]
to navigate to the tutorial directory.
[01:57]
And here now I'm going to create a file
[01:58]
called main.py. Now I'm going to start
[02:00]
by importing async io and creating a
[02:03]
so-called co- routine. So a co- routine
[02:05]
is basically a program component that
[02:06]
can be suspended and resumed. So we can
[02:08]
pause this. We can continue with this.
[02:11]
And we define it by saying async defaf
[02:13]
and then the name IO task. So an
[02:15]
asynchronous function essentially. In
[02:17]
this case, this one takes name, delay,
[02:19]
and number of iterations as a parameter.
[02:22]
Then we have a couple of iterations in
[02:24]
this loop here. And we just print the
[02:26]
task name and the current iteration just
[02:28]
so we can keep track of what is actually
[02:30]
happening. And the key thing here is
[02:32]
awaiting the async io. Call. So this is
[02:35]
just a placeholder. You could have
[02:37]
anything here awaiting something that is
[02:39]
asynchronous. So this could be also
[02:41]
waiting for a response from a server.
[02:43]
Basically just any downtime that can be
[02:46]
used in this single thread that we're
[02:47]
running. Now asynchronous programming
[02:49]
runs in a so-called event loop. We have
[02:52]
one thread so we don't have any
[02:53]
concurrent I mean we do have concurrency
[02:55]
but we don't have any parallel
[02:57]
execution. We don't have multiple
[02:58]
threads. We don't have multiple
[03:00]
processes. We have one thread and the
[03:02]
event loop basically switches between
[03:04]
the co- routines. So in this case, what
[03:06]
we're saying here is we're saying print
[03:09]
a statement and then give back control
[03:11]
yield back to the event loop and allow
[03:14]
it to do something else while we're
[03:16]
doing this. So we're basically saying
[03:18]
sleep for whatever we pass as delay
[03:21]
seconds and then go back here. So every
[03:24]
iteration each iteration here is going
[03:26]
to call this await async io sleep which
[03:29]
means we're giving back control to the
[03:30]
event loop and the event loop can then
[03:32]
determine which of the other co-
[03:34]
routines are capable of resuming. So we
[03:37]
can see that this works by defining an
[03:39]
asynchronous main function. What we do
[03:41]
here is we measure the time of the
[03:43]
executions. We have one time here an
[03:45]
async io gather call. So we're calling
[03:48]
the gather function and we're passing
[03:51]
here three tasks called A, B and C with
[03:53]
different delays but the same number of
[03:55]
iterations. And these are going to be
[03:58]
executed asynchronously. So concurrently
[04:00]
as three co- routines which basically
[04:02]
means when A is sleeping we can do B.
[04:05]
When A and B are sleeping we can do C
[04:07]
and so on. So we can switch back and
[04:09]
forth because we have this downtime this
[04:11]
idle time. Uh in addition to that down
[04:14]
below here we have three separate await
[04:16]
statements. So we await three tasks in a
[04:18]
row. So this is serially. This is not
[04:20]
concurrently. We're not using the gather
[04:22]
function. And this basically means task
[04:25]
A has to be executed. Then task B has to
[04:28]
be executed. Task C has to be executed.
[04:30]
And then we're done. Now of course here
[04:32]
I also need to import time and also of
[04:35]
course we need to run the main function
[04:37]
here. We do that by starting an event
[04:39]
loop by creating an event loop with
[04:41]
async io run. So we do async io run and
[04:44]
we pass main. But we don't pass main as
[04:47]
a function. So as a reference to the
[04:49]
function, we actually call main and we
[04:51]
do that in async.io run. So we're
[04:53]
actually using parenthesis in here. So
[04:56]
when I run this now, you can see we have
[04:58]
a, b, and c being executed concurrently.
[05:01]
So this happens um yeah at the same time
[05:04]
basically 4.5 seconds. Whereas if I do
[05:07]
that separately, we can see that we have
[05:09]
first a then b then c and this is going
[05:12]
to take much longer 11 seconds. Now,
[05:15]
this also happens if we're not awaiting.
[05:18]
Await is the keyword that returns
[05:20]
control back to the event loop. So, if I
[05:22]
instead cause some downtime here with
[05:24]
time. Which is perfectly fine. I can do
[05:27]
that. If I say time.sleep delay instead
[05:30]
of async io sleep delay, this doesn't
[05:33]
work anymore because now I'm never
[05:34]
returning control back to the event
[05:36]
loop. I now basically say there is some
[05:39]
downtime. But since we're not using
[05:40]
multi-threading here, what is actually
[05:42]
happening is we're just waiting. we're
[05:44]
blocking uh the threat. So we're just
[05:47]
waiting for this to finish before we can
[05:49]
move on. So if you actually want to give
[05:51]
control back to the event loop, you have
[05:53]
to use await. And in this case, you
[05:55]
would have to use async io. Another
[05:58]
thing that we can do is we can run tasks
[06:00]
in the background and we can return
[06:02]
them, save them into a variable and then
[06:04]
await them at some point later in the
[06:05]
function. So here for example, I have
[06:07]
this background task which prints
[06:09]
running then waits for 5 seconds then
[06:11]
prints finishing. And what I can do here
[06:13]
is I can do async.io.create task with
[06:17]
background task being called in here. Uh
[06:19]
this returns then the task instance.
[06:22]
Whatever happens afterwards is executed
[06:24]
immediately. So this print statement for
[06:26]
example. But then I can also await the
[06:28]
task and I can say okay don't continue
[06:31]
until this is done and then print the
[06:34]
final statement. And of course
[06:35]
everything that happens after async io
[06:37]
run also has to wait for all of this to
[06:40]
finish. So if I run this you can see
[06:42]
continuing immediately um even before
[06:44]
running is being printed and then only
[06:47]
when this background task is finished
[06:49]
because we're awaiting it here only then
[06:51]
do we get but for this we need to wait
[06:53]
and this waits two what can also be
[06:55]
interesting is using the wait function
[06:57]
in this case here we have again a very
[06:59]
simple setup we have two co- routines
[07:01]
print statement waiting time print
[07:03]
statement return value here with 2
[07:05]
seconds here with 5 seconds and then we
[07:08]
have them as two tasks and Then we use
[07:10]
the weight function not the gather
[07:12]
function. This allows us to specify a
[07:15]
return condition. So in this case we do
[07:17]
done and pending await async io.we and
[07:21]
then we specify here return when async
[07:23]
io first completed. What this basically
[07:25]
means is that this whole thing is going
[07:28]
to return. So we're going to stop
[07:30]
waiting when one of them returns. So
[07:32]
when the first one returns we're going
[07:34]
to continue with the code and this is
[07:36]
going to return two things. It's going
[07:37]
to return the list of the tasks that are
[07:39]
finished. So that are done and the list
[07:42]
of the tasks that are not finished yet.
[07:44]
So we can also print all of that after
[07:46]
this is being awaited. We can print the
[07:48]
finished tasks and the pending tasks.
[07:50]
And then in the end we can also do
[07:52]
another await asai await pending to wait
[07:55]
for the remaining tasks. So let's run
[07:57]
this now. You can see one start two
[08:00]
start then one finishes one end. You can
[08:02]
see the finished tasks are uh result is
[08:05]
equal to one done and then we have the
[08:07]
pending tasks which doesn't have a
[08:10]
result yet. So we have a future and at
[08:12]
some point then this also finishes and
[08:14]
we get to end. Now if you don't want to
[08:17]
wait indefinitely, you can also use the
[08:18]
wait for function. This allows us to
[08:20]
specify a timeout. In this case we have
[08:22]
a long operation taking 5 seconds and we
[08:25]
only allow for 2 seconds by using wait
[08:27]
four. In the case that these two seconds
[08:29]
are surpassed we get a timeout error. we
[08:32]
can catch that and handle it. Uh but in
[08:34]
this case, we're just going to print
[08:36]
took too long. So if I run this, you're
[08:37]
going to see one, two, took too long
[08:40]
because this takes 5 seconds and this
[08:43]
takes 2 seconds. Of course, this only
[08:45]
works with a wait because we need to
[08:46]
pass control back to the event loop. Uh
[08:48]
it doesn't work if I use time. Because
[08:50]
then it's going to block the threat. And
[08:52]
that's basically it. There's of course
[08:54]
much more to cover. Manual stuff you can
[08:56]
do with the event loop, task groups,
[08:58]
shielding, and so on. There's much more
[08:59]
to cover in general when it comes to
[09:01]
concurrency in Python. If you want to
[09:03]
have more detailed tutorials, let me
[09:04]
know in the comment section down below.
[09:06]
So, that's it for today's video. I hope
[09:08]
you enjoyed it and hope you learned
[09:09]
something. If so, let me know by hitting
[09:11]
a like button and leaving a comment in
[09:12]
the comment section down below. Also,
[09:14]
don't forget to check out the similar
[09:15]
videos I already have on multi-threading
[09:17]
and multipprocessing. And of course,
[09:19]
don't forget to subscribe to this
[09:20]
channel and hit the notification bell to
[09:22]
not miss a single future video for free.
[09:24]
Other than that, thank you much for
[09:25]
watching. See you in the next video and
[09:27]
bye.