AI Summary
This video provides a quick introduction to asynchronous programming in Python using AsyncIO. It explains how to create coroutines, use the event loop, and manage concurrent tasks with `await`, `gather`, and `wait` functions.
Chapters
The video aims to cover asynchronous programming in Python with AsyncIO as quickly as possible, without deep diving into details.
Three major ways: asynchronous programming, multi-threading, and multiprocessing. Multiprocessing for CPU-bound tasks, multi-threading for I/O-bound tasks without manual control, and async for I/O-bound tasks with manual control.
A coroutine is defined with `async def` and can be suspended/resumed. Example: `async def io_task(name, delay, iterations)`.
`await asyncio.sleep(delay)` yields control back to the event loop, allowing other coroutines to run during idle time.
`asyncio.gather` runs multiple coroutines concurrently. Example: `asyncio.gather(task_a, task_b, task_c)` completes in ~4.5 seconds vs 11 seconds serially.
Using `time.sleep` instead of `await asyncio.sleep` blocks the thread and prevents concurrency. `await` is essential to return control to the event loop.
`asyncio.create_task` schedules a coroutine to run in the background. Example: `task = asyncio.create_task(background_task())` allows other code to run before awaiting the task.
`asyncio.wait` can specify return conditions like `FIRST_COMPLETED`. Returns sets of done and pending tasks. Example: `done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)`.
`asyncio.wait_for` adds a timeout. If exceeded, raises `asyncio.TimeoutError`. Example: `await asyncio.wait_for(long_operation(), timeout=2)`.
AsyncIO enables efficient concurrency for I/O-bound tasks using a single thread and an event loop. Key functions like `gather`, `wait`, and `wait_for` provide flexible control over coroutine execution.
Mentioned in this Video
Tutorial Checklist
Study Flashcards (7)
What is a coroutine in Python?
easy
Click to reveal answer
What is a coroutine in Python?
A program component that can be suspended and resumed, defined with `async def`.
2:03
What does `await asyncio.sleep(delay)` do?
medium
Click to reveal answer
What does `await asyncio.sleep(delay)` do?
It yields control back to the event loop, allowing other coroutines to run during the sleep period.
2:32
How does `asyncio.gather` differ from sequential awaits?
medium
Click to reveal answer
How does `asyncio.gather` differ from sequential awaits?
`gather` runs coroutines concurrently, while sequential awaits run them one after another.
3:39
What happens if you use `time.sleep` inside a coroutine instead of `await asyncio.sleep`?
hard
Click to reveal answer
What happens if you use `time.sleep` inside a coroutine instead of `await asyncio.sleep`?
It blocks the thread and prevents the event loop from switching to other coroutines, eliminating concurrency.
5:15
How do you create a background task in AsyncIO?
medium
Click to reveal answer
How do you create a background task in AsyncIO?
Use `asyncio.create_task(coroutine())` to schedule it and later `await` the task object.
6:00
What does `asyncio.wait` with `return_when=asyncio.FIRST_COMPLETED` return?
hard
Click to reveal answer
What does `asyncio.wait` with `return_when=asyncio.FIRST_COMPLETED` return?
It returns two sets: done tasks and pending tasks, after the first task completes.
6:55
How do you add a timeout to an async operation?
medium
Click to reveal answer
How do you add a timeout to an async operation?
Use `asyncio.wait_for(coroutine(), timeout=seconds)` and catch `asyncio.TimeoutError` if exceeded.
8:17
💡 Key Takeaways
Three Concurrency Methods
Clearly distinguishes when to use async vs multi-threading vs multiprocessing in Python.
0:42Coroutine Definition
Introduces the core concept of a coroutine as a suspendable component.
2:03gather vs Sequential Execution
Demonstrates the performance benefit of concurrency with a concrete example (4.5s vs 11s).
3:39Await is Essential
Highlights a common pitfall: using blocking calls like `time.sleep` kills concurrency.
5:15wait with FIRST_COMPLETED
Shows a practical pattern for handling tasks as they complete, useful for time-sensitive operations.
6:55Full Transcript
[00:00] Today we're going to learn about
[00:01] asynchronous programming in Python with
[00:03] Async IO as quickly as possible. So let
[00:05] us get right into it.
[00:11] [Music]
[00:15] >> All right, so as always when it comes to
[00:17] these tutorials where I try to cover
[00:18] something as quickly as possible, this
[00:20] is not going to be a deep dive. We're
[00:22] not going to go into a lot of details. I
[00:24] wouldn't even call it a crash course.
[00:25] It's more like me giving you a very very
[00:27] quick introduction into the topic and
[00:29] then you can continue to study it on
[00:31] your own or if you want to you can leave
[00:33] me a comment in the comment section down
[00:34] below and let me know that you're
[00:36] interested in a more detailed course so
[00:38] maybe I can do that as well on my
[00:39] channel but asynchronous programming
[00:42] belongs to the category of concurrent
[00:44] programming in Python and there are
[00:45] three major ways to do concurrent
[00:47] programming in Python one is
[00:49] asynchronous programming another one is
[00:51] multi-threading and another one is
[00:53] multipprocessing for the last two I
[00:55] already have two videos on my channel
[00:57] similar to this one where I cover them
[00:59] as quickly as possible. So you can take
[01:00] a look at them if you want to. And today
[01:03] we're going to talk about asynchronous
[01:04] programming. Now in a nutshell, you want
[01:06] to use multipprocessing when you have a
[01:08] lot of CPUbound tasks that you want to
[01:10] parallelize. So heavy computations that
[01:13] you want to do simultaneously on
[01:15] multiple CPU cores. You actually want to
[01:17] have multiple processes. Uh you don't
[01:19] want to be limited by the global
[01:20] interpreter log. Multi-threading is a
[01:23] little bit more I wouldn't necessarily
[01:25] say exotic, but it's a bit odd in Python
[01:28] because you have the global interpreter
[01:29] lock and you don't have real
[01:30] multi-threading unless you release the
[01:32] global interpreter lock. But
[01:34] essentially, you want to use it when you
[01:35] don't want to manually handle the
[01:38] control. So you don't want to manually
[01:39] switch between the different threads and
[01:41] you also maybe have to work with
[01:43] something that doesn't support
[01:44] asynchronous programming. But I want I
[01:46] don't want to talk about this too much.
[01:47] I want to focus on asynchronous
[01:49] programming today. uh which is the topic
[01:51] of this video. So let us go into our
[01:53] coding directory. In my case, I'm going
[01:55] to navigate to the tutorial directory.
[01:57] And here now I'm going to create a file
[01:58] called main.py. Now I'm going to start
[02:00] by importing async io and creating a
[02:03] so-called co- routine. So a co- routine
[02:05] is basically a program component that
[02:06] can be suspended and resumed. So we can
[02:08] pause this. We can continue with this.
[02:11] And we define it by saying async defaf
[02:13] and then the name IO task. So an
[02:15] asynchronous function essentially. In
[02:17] this case, this one takes name, delay,
[02:19] and number of iterations as a parameter.
[02:22] Then we have a couple of iterations in
[02:24] this loop here. And we just print the
[02:26] task name and the current iteration just
[02:28] so we can keep track of what is actually
[02:30] happening. And the key thing here is
[02:32] awaiting the async io. Call. So this is
[02:35] just a placeholder. You could have
[02:37] anything here awaiting something that is
[02:39] asynchronous. So this could be also
[02:41] waiting for a response from a server.
[02:43] Basically just any downtime that can be
[02:46] used in this single thread that we're
[02:47] running. Now asynchronous programming
[02:49] runs in a so-called event loop. We have
[02:52] one thread so we don't have any
[02:53] concurrent I mean we do have concurrency
[02:55] but we don't have any parallel
[02:57] execution. We don't have multiple
[02:58] threads. We don't have multiple
[03:00] processes. We have one thread and the
[03:02] event loop basically switches between
[03:04] the co- routines. So in this case, what
[03:06] we're saying here is we're saying print
[03:09] a statement and then give back control
[03:11] yield back to the event loop and allow
[03:14] it to do something else while we're
[03:16] doing this. So we're basically saying
[03:18] sleep for whatever we pass as delay
[03:21] seconds and then go back here. So every
[03:24] iteration each iteration here is going
[03:26] to call this await async io sleep which
[03:29] means we're giving back control to the
[03:30] event loop and the event loop can then
[03:32] determine which of the other co-
[03:34] routines are capable of resuming. So we
[03:37] can see that this works by defining an
[03:39] asynchronous main function. What we do
[03:41] here is we measure the time of the
[03:43] executions. We have one time here an
[03:45] async io gather call. So we're calling
[03:48] the gather function and we're passing
[03:51] here three tasks called A, B and C with
[03:53] different delays but the same number of
[03:55] iterations. And these are going to be
[03:58] executed asynchronously. So concurrently
[04:00] as three co- routines which basically
[04:02] means when A is sleeping we can do B.
[04:05] When A and B are sleeping we can do C
[04:07] and so on. So we can switch back and
[04:09] forth because we have this downtime this
[04:11] idle time. Uh in addition to that down
[04:14] below here we have three separate await
[04:16] statements. So we await three tasks in a
[04:18] row. So this is serially. This is not
[04:20] concurrently. We're not using the gather
[04:22] function. And this basically means task
[04:25] A has to be executed. Then task B has to
[04:28] be executed. Task C has to be executed.
[04:30] And then we're done. Now of course here
[04:32] I also need to import time and also of
[04:35] course we need to run the main function
[04:37] here. We do that by starting an event
[04:39] loop by creating an event loop with
[04:41] async io run. So we do async io run and
[04:44] we pass main. But we don't pass main as
[04:47] a function. So as a reference to the
[04:49] function, we actually call main and we
[04:51] do that in async.io run. So we're
[04:53] actually using parenthesis in here. So
[04:56] when I run this now, you can see we have
[04:58] a, b, and c being executed concurrently.
[05:01] So this happens um yeah at the same time
[05:04] basically 4.5 seconds. Whereas if I do
[05:07] that separately, we can see that we have
[05:09] first a then b then c and this is going
[05:12] to take much longer 11 seconds. Now,
[05:15] this also happens if we're not awaiting.
[05:18] Await is the keyword that returns
[05:20] control back to the event loop. So, if I
[05:22] instead cause some downtime here with
[05:24] time. Which is perfectly fine. I can do
[05:27] that. If I say time.sleep delay instead
[05:30] of async io sleep delay, this doesn't
[05:33] work anymore because now I'm never
[05:34] returning control back to the event
[05:36] loop. I now basically say there is some
[05:39] downtime. But since we're not using
[05:40] multi-threading here, what is actually
[05:42] happening is we're just waiting. we're
[05:44] blocking uh the threat. So we're just
[05:47] waiting for this to finish before we can
[05:49] move on. So if you actually want to give
[05:51] control back to the event loop, you have
[05:53] to use await. And in this case, you
[05:55] would have to use async io. Another
[05:58] thing that we can do is we can run tasks
[06:00] in the background and we can return
[06:02] them, save them into a variable and then
[06:04] await them at some point later in the
[06:05] function. So here for example, I have
[06:07] this background task which prints
[06:09] running then waits for 5 seconds then
[06:11] prints finishing. And what I can do here
[06:13] is I can do async.io.create task with
[06:17] background task being called in here. Uh
[06:19] this returns then the task instance.
[06:22] Whatever happens afterwards is executed
[06:24] immediately. So this print statement for
[06:26] example. But then I can also await the
[06:28] task and I can say okay don't continue
[06:31] until this is done and then print the
[06:34] final statement. And of course
[06:35] everything that happens after async io
[06:37] run also has to wait for all of this to
[06:40] finish. So if I run this you can see
[06:42] continuing immediately um even before
[06:44] running is being printed and then only
[06:47] when this background task is finished
[06:49] because we're awaiting it here only then
[06:51] do we get but for this we need to wait
[06:53] and this waits two what can also be
[06:55] interesting is using the wait function
[06:57] in this case here we have again a very
[06:59] simple setup we have two co- routines
[07:01] print statement waiting time print
[07:03] statement return value here with 2
[07:05] seconds here with 5 seconds and then we
[07:08] have them as two tasks and Then we use
[07:10] the weight function not the gather
[07:12] function. This allows us to specify a
[07:15] return condition. So in this case we do
[07:17] done and pending await async io.we and
[07:21] then we specify here return when async
[07:23] io first completed. What this basically
[07:25] means is that this whole thing is going
[07:28] to return. So we're going to stop
[07:30] waiting when one of them returns. So
[07:32] when the first one returns we're going
[07:34] to continue with the code and this is
[07:36] going to return two things. It's going
[07:37] to return the list of the tasks that are
[07:39] finished. So that are done and the list
[07:42] of the tasks that are not finished yet.
[07:44] So we can also print all of that after
[07:46] this is being awaited. We can print the
[07:48] finished tasks and the pending tasks.
[07:50] And then in the end we can also do
[07:52] another await asai await pending to wait
[07:55] for the remaining tasks. So let's run
[07:57] this now. You can see one start two
[08:00] start then one finishes one end. You can
[08:02] see the finished tasks are uh result is
[08:05] equal to one done and then we have the
[08:07] pending tasks which doesn't have a
[08:10] result yet. So we have a future and at
[08:12] some point then this also finishes and
[08:14] we get to end. Now if you don't want to
[08:17] wait indefinitely, you can also use the
[08:18] wait for function. This allows us to
[08:20] specify a timeout. In this case we have
[08:22] a long operation taking 5 seconds and we
[08:25] only allow for 2 seconds by using wait
[08:27] four. In the case that these two seconds
[08:29] are surpassed we get a timeout error. we
[08:32] can catch that and handle it. Uh but in
[08:34] this case, we're just going to print
[08:36] took too long. So if I run this, you're
[08:37] going to see one, two, took too long
[08:40] because this takes 5 seconds and this
[08:43] takes 2 seconds. Of course, this only
[08:45] works with a wait because we need to
[08:46] pass control back to the event loop. Uh
[08:48] it doesn't work if I use time. Because
[08:50] then it's going to block the threat. And
[08:52] that's basically it. There's of course
[08:54] much more to cover. Manual stuff you can
[08:56] do with the event loop, task groups,
[08:58] shielding, and so on. There's much more
[08:59] to cover in general when it comes to
[09:01] concurrency in Python. If you want to
[09:03] have more detailed tutorials, let me
[09:04] know in the comment section down below.
[09:06] So, that's it for today's video. I hope
[09:08] you enjoyed it and hope you learned
[09:09] something. If so, let me know by hitting
[09:11] a like button and leaving a comment in
[09:12] the comment section down below. Also,
[09:14] don't forget to check out the similar
[09:15] videos I already have on multi-threading
[09:17] and multipprocessing. And of course,
[09:19] don't forget to subscribe to this
[09:20] channel and hit the notification bell to
[09:22] not miss a single future video for free.
[09:24] Other than that, thank you much for
[09:25] watching. See you in the next video and
[09:27] bye.