[00:00]
how's it going guys in today's lesson
[00:01]
we're going to be going over
[00:03]
asynchronous programming in python and
[00:06]
we're going to be using async io which
[00:08]
is a package included with python the
[00:11]
vanilla version
[00:12]
so to get started we're going to go
[00:14]
ahead and import async io and in the
[00:18]
first example we're going to try to
[00:20]
simulate an api request so we can see
[00:23]
how this can actually be used in a real
[00:25]
world project so actually i want to go
[00:27]
ahead and create the api first so here
[00:30]
we're going to go ahead and create a
[00:32]
package called api and we need to go
[00:34]
ahead and also import async io here
[00:38]
because we're going to be using one of
[00:39]
its basic features now the first thing
[00:42]
you need to note is that every time you
[00:43]
create an asynchronous function it's
[00:45]
going to start with the keyword async
[00:48]
followed by your function name which in
[00:51]
this example just going to be called
[00:52]
fetch data
[00:54]
and it's going to return some data which
[00:55]
will just say of type string for this
[00:57]
example
[00:59]
now we can go ahead and print something
[01:01]
such as fetching data so we know that
[01:04]
the api request is being called and then
[01:06]
we're going to go ahead and create a
[01:08]
delay so inside here we're going to go
[01:10]
ahead and call a weight and a weight is
[01:12]
a way of forcing the program to wait for
[01:16]
the current function to finish because
[01:18]
in asynchronous programming if you have
[01:20]
a bunch of async functions and you call
[01:22]
them all at the same time they're going
[01:24]
to all work at the same time but one
[01:26]
very important feature in asynchronous
[01:28]
programming is being able to wait for a
[01:30]
certain line of code and forcing the
[01:32]
program to wait for that to complete
[01:34]
before moving on in that asynchronous
[01:36]
function
[01:37]
so in this example we want to call async
[01:40]
io dot sleep and we're going to put a
[01:43]
delay of 2.5 seconds so a weight just
[01:46]
tells the program that
[01:48]
you need to wait for this before
[01:49]
executing the next statement
[01:51]
which is just going to say data fetched
[01:54]
and down here we're going to return
[01:57]
the api
[01:59]
data so just by creating this async
[02:02]
function we can now place it anywhere
[02:03]
and it's going to run asynchronously
[02:05]
which means if we want to create an api
[02:07]
request
[02:08]
we can do that and we can do other work
[02:10]
at the same time
[02:12]
so
[02:13]
we're going to go back to our main.pi
[02:15]
file
[02:16]
and inside here we're going to go ahead
[02:18]
and import
[02:20]
api and we can close the sidebar
[02:23]
so what we want to do inside here is now
[02:25]
create a program that sends data to
[02:28]
users after it fetches the data
[02:30]
so to do this we're going to go ahead
[02:32]
and create an async function which is
[02:35]
going to be called send data
[02:37]
and it's going to take a to user which
[02:39]
will be of type string in this example
[02:42]
and here we'll type in print formatted
[02:44]
string sending
[02:46]
data to
[02:48]
and we insert the name inside there
[02:51]
then we're going to call await again and
[02:53]
async dot io sleep and we want to give
[02:57]
it a two second delay now we can go
[02:59]
ahead and print that the data was sent
[03:02]
to the person that we sent it to so
[03:05]
that's a very simple sent data function
[03:07]
which of course you would replace with
[03:09]
your own network request now we're going
[03:11]
to make sure that this all runs
[03:12]
asynchronously so we're going to call
[03:14]
async def main which is going to be our
[03:18]
main program
[03:19]
and inside here the first thing we want
[03:21]
to do is call our data which is going to
[03:23]
be
[03:24]
an await of api dot fetch data so it's
[03:28]
important that we await this because
[03:29]
this is the data we want to get back and
[03:31]
we don't want to progress with the
[03:33]
program until we've received this data
[03:35]
that's why we use a weight here and we
[03:38]
can also go ahead and print the data
[03:40]
which is going to be of type data
[03:42]
which is going to be the data now let's
[03:44]
go ahead and try to send this to a
[03:46]
certain user so here we're going to go
[03:48]
ahead and call await and we're going to
[03:50]
send the data to mario and let's go
[03:53]
ahead and run this just to see what
[03:54]
we've done so far
[03:56]
so inside here to run an asynchronous
[03:59]
program you need to call async io dot
[04:01]
run
[04:02]
and insert your master program which for
[04:04]
us is just going to be main so just like
[04:07]
that we can go ahead and click on run
[04:09]
and it's going to fetch the data and as
[04:11]
soon as it fetches the data it's going
[04:13]
to display the data it's going to try to
[04:15]
send the data to mario and it's going to
[04:17]
send the data to mario
[04:19]
so so far everything was quite
[04:21]
synchronous it fetched the data it
[04:24]
assigned the data to the data value and
[04:26]
sent it to mario now
[04:28]
that's not really the asynchronous
[04:30]
programming part even though our code is
[04:33]
asynchronous we haven't used it in an
[04:35]
asynchronous way so what i want to
[04:37]
demonstrate to you here is that we can
[04:39]
send this to multiple users at the same
[04:42]
time because if we just call a wait each
[04:45]
time it's going to call it one after the
[04:48]
other and that's not really what we
[04:49]
wanted to do we want to send it to luigi
[04:52]
and mario at the exact same time so how
[04:55]
would we accomplish this and the easiest
[04:57]
way to do this is to use the gather
[05:00]
function in async io which takes as many
[05:03]
functions as you want to insert so
[05:06]
inside here we're going to send data to
[05:08]
mario for example and we're also going
[05:10]
to send data
[05:12]
to
[05:12]
luigi so we're going to use these two
[05:14]
functions at the exact same time
[05:17]
and it might be easier for you to create
[05:19]
a list of functions that are
[05:22]
asynchronous if you want to do that and
[05:25]
then just pass the list inside here but
[05:27]
for this example it's quite easy just to
[05:28]
insert two functions
[05:30]
and as soon as we run the program it's
[05:33]
going to fetch the data and then we're
[05:35]
going to retrieve the data and it's
[05:36]
going to send to mario and to luigi at
[05:38]
the exact same moment instead of sending
[05:41]
it to mario first and then to luigi we
[05:43]
were able to send it to both of them at
[05:45]
the exact same time so that was the
[05:47]
first example of using asynchronous
[05:49]
programming now next we're going to go
[05:51]
ahead and create a function that creates
[05:53]
a thousand functions and runs them all
[05:55]
asynchronously just to show you one more
[05:58]
time how we can create lots of functions
[06:00]
that run asynchronously so we're going
[06:02]
to go here and delete all of that and
[06:05]
we're still going to keep the main over
[06:06]
here
[06:07]
but what we're going to do as well is
[06:09]
create a async function that is called
[06:12]
kill time
[06:14]
and it's going to take a number
[06:16]
now we're going to print that this is
[06:18]
running and we're going to insert the
[06:21]
number here so we know which function it
[06:23]
is and we're going to go ahead and await
[06:26]
async io
[06:28]
dot sleep
[06:29]
for let's say one second and then we
[06:31]
will print
[06:33]
finished
[06:34]
with the number so that's the only
[06:37]
function we're going to create and it's
[06:39]
quite simple it just tells us that we're
[06:40]
running that function and that we're
[06:43]
done with that function so we're going
[06:45]
to run this 1000 times
[06:47]
at the same time and we need to go ahead
[06:49]
and create our async
[06:52]
main function and inside here we're
[06:54]
going to print that we've started the
[06:56]
process now we're going to create a list
[06:58]
of tasks
[07:00]
which will be an empty array and then
[07:02]
for i
[07:04]
in range
[07:05]
1
[07:07]
to 1000 plus 1 we're going to go ahead
[07:10]
and say list of tasks dot append and we
[07:13]
want to append kill time plus the
[07:16]
current iteration so we can tell which
[07:18]
one we're on currently and before we do
[07:21]
anything we're going to give this a two
[07:23]
second delay so async io dot sleep and
[07:26]
two seconds and we also need to call
[07:28]
await await.asyncio.ganda
[07:32]
and as i mentioned earlier you can place
[07:34]
as many functions as you want in here
[07:37]
but if you want to create a list of
[07:38]
functions you need to insert the
[07:40]
asterisk which says we're going to
[07:42]
insert
[07:43]
as many arguments as we want so here
[07:46]
we're going to go ahead and insert the
[07:47]
list of tasks and then we can go ahead
[07:50]
and print done now when we go ahead and
[07:52]
run this program
[07:54]
it's going to say started and then it's
[07:56]
going to run all of them at the exact
[07:58]
same time and finish them at the exact
[08:00]
same time then it's going to say done so
[08:03]
here we successfully ran 1000 functions
[08:06]
at the exact same time and as you can
[08:08]
see it's quite a lot so
[08:10]
asyncio.gather is crazy powerful in that
[08:13]
respect that we can run as many
[08:15]
functions as we want even in this
[08:17]
demonstration where we ran 1000 it still
[08:20]
was able to handle that so the sky is
[08:22]
the limit now the final part of this
[08:25]
tutorial will be explaining how you can
[08:27]
use tasks in async io so that you can
[08:30]
start them and you can cancel them and
[08:32]
you can check for the result so to do
[08:34]
this we're just going to get rid of all
[08:37]
of this
[08:37]
create an async dev main and inside here
[08:41]
the first thing we want to do is create
[08:42]
a task
[08:44]
and the task is going to be async io dot
[08:47]
create task
[08:48]
and inside here you insert whatever task
[08:51]
you want so for this example we're going
[08:53]
to create api fetch data as our main
[08:56]
task so as soon as you create this task
[08:59]
it actually starts the task so right now
[09:01]
it has been called as soon as you've
[09:03]
created the task and what we're going to
[09:05]
do is create a try and accept block and
[09:08]
inside here we're going to add something
[09:09]
called result which is going to be the
[09:11]
weight of task and accept is just going
[09:14]
to be empty for now we're just going to
[09:16]
pass and let's actually go ahead and
[09:18]
print the result as well
[09:21]
so if we go ahead and click on run it's
[09:24]
going to fetch the data and as soon as
[09:26]
it retrieves the data it's going to say
[09:28]
that we've fetched this api data so
[09:31]
that's very similar to just creating a
[09:33]
normal awaits
[09:35]
except this time we inserted it inside
[09:37]
here now the benefit of creating a task
[09:40]
is that you have some extra context
[09:42]
options such as task dot cancel now if
[09:46]
we actually go ahead and run this
[09:47]
nothing's going to happen and it's going
[09:49]
to throw an exception but instead of
[09:51]
accepting nothing which is very bad
[09:53]
practice we're going to go ahead and
[09:56]
accept async io canceled error
[10:00]
and inside here we'll just type
[10:02]
canceled
[10:03]
request was cancelled
[10:05]
and while we're here we're going to
[10:06]
create another exception because i
[10:08]
believe there's only two exceptions for
[10:10]
this example and the second one is going
[10:12]
to be a timeout error and for the second
[10:15]
exception we're just going to print the
[10:17]
same thing except you will say timeout
[10:20]
request
[10:21]
took
[10:22]
too long so now if we run the program
[10:25]
we're going to get the error that the
[10:27]
request was cancelled so it tried to
[10:29]
make the request and wild was
[10:31]
calculating it said okay you're taking
[10:33]
too long so we're going to cancel you
[10:35]
and you can place this at any moment in
[10:37]
your program as long as the task is
[10:39]
running it will be able to cancel it in
[10:41]
time and there's also another check you
[10:43]
can say
[10:44]
if task
[10:46]
is canceled
[10:48]
then you can do the following check this
[10:50]
is going to return a boolean
[10:52]
so here we can just say print task dot
[10:55]
canceled which is going to return true
[10:57]
but we also have to go ahead and call
[10:59]
async io dot sleep and give it a 0.5
[11:02]
delay because sometimes this happens so
[11:04]
fast that task.canceled is not going to
[11:07]
be able to register the cancel in enough
[11:09]
time
[11:10]
and it's not going to be able to check
[11:12]
for that so it's good to give it a delay
[11:14]
sometimes
[11:15]
so the program can catch up with what
[11:17]
it's trying to check for
[11:19]
so if we go ahead and run this
[11:21]
we're going to get true because the task
[11:23]
was cancelled since we cancelled it up
[11:25]
here
[11:26]
and it's going to give us the same error
[11:28]
as from earlier now let's go ahead and
[11:30]
remove all of this
[11:32]
now another very good method provided by
[11:34]
task is the task.done method and this
[11:38]
returns true if we finish the task if
[11:40]
all the code was executed successfully
[11:43]
in the task this will return true
[11:45]
so what you can do is go ahead and type
[11:47]
in if task.done
[11:49]
then we want to go ahead and print the
[11:52]
task dot result
[11:54]
if you try to print the task.result
[11:56]
ahead of time and there's nothing there
[11:58]
it's going to give you an error
[12:01]
so we're going to go ahead and remove
[12:03]
all of this
[12:04]
and since we know very well that the
[12:06]
task takes 2.5 seconds we're going to go
[12:09]
ahead and call async io dot sleep for
[12:12]
three seconds
[12:13]
so here we can go ahead and type in a
[12:15]
wait
[12:17]
and as soon as it finishes getting the
[12:20]
data it's going to print the data out
[12:22]
here otherwise
[12:24]
if we go ahead and just give it two
[12:26]
seconds instead of 2.5 it's not going to
[12:29]
have time to finish and it's not going
[12:31]
to be able to print the data so this is
[12:33]
a good way to verify that you actually
[12:35]
have a result now the final method i
[12:38]
want to show you inside here is a way to
[12:41]
give this a timeout maybe you don't want
[12:43]
to wait so long for the task there's a
[12:45]
way to say that after two seconds we're
[12:48]
going to give up and to do this we're
[12:50]
going to go ahead and await async io dot
[12:54]
wait for
[12:55]
and as you can see inside here we can
[12:56]
insert a task
[12:58]
and we can insert a timeout so let's say
[13:01]
after two seconds we want this just to
[13:03]
give up
[13:04]
now when we go ahead and run this
[13:06]
program it's going to try to fetch the
[13:08]
data but it's going to take too long
[13:10]
because this function over here takes
[13:12]
2.5 seconds and the timeout is set to 2.
[13:16]
so if you're making an api request you
[13:18]
might set this to 10 seconds because you
[13:19]
don't want the user to wait too long it
[13:22]
might mean they don't have internet it
[13:23]
might mean something else so the timer
[13:25]
just gives you a way of handling
[13:27]
requests that take far too long and
[13:29]
everything that takes far too long is
[13:31]
going to be handled in this accept block
[13:33]
but anyways guys that's actually all i
[13:35]
wanted to cover in today's lesson we
[13:37]
covered how we can create some really
[13:39]
cool asynchronous requests and those
[13:41]
were essentially the basics of how async
[13:44]
i o works i'm going to leave a link to
[13:46]
the documentation down below because
[13:48]
it's really well written and easy to
[13:50]
read so in case you want to learn about
[13:52]
some more particulars regarding this
[13:54]
definitely check out the documentation
[13:57]
but with that being said thanks for
[13:58]
watching and i'll see you in the next
[14:00]
lesson