[00:00]
Hello guys, today let's talk about
[00:01]
Laravel Q's. I will try to take this
[00:03]
very complex topic and explain the main
[00:05]
things that you need to know about Q's
[00:07]
in roughly 10 to 15 minutes. And this is
[00:09]
kind of a challenge I decided for
[00:10]
myself. Take some complex topic in
[00:13]
Laravel and try to explain it in 10 to
[00:15]
15 minutes. So you will see more of
[00:16]
these videos in the future. How did I go
[00:18]
with this simplification? Shoot in the
[00:20]
comments below. So today we'll talk
[00:22]
about how to put the email notification
[00:24]
in a queue with this example and I'll
[00:26]
show you how different the experience is
[00:28]
with and without Q's. Then the second
[00:30]
example would be how to create a Q job
[00:33]
to build the PDF invoice for the order
[00:35]
and the link to code repository for both
[00:37]
examples will be in the description
[00:38]
below. Also we'll talk about how to deal
[00:40]
with Q job failures, how to generally
[00:43]
run the Q and what should be the Q
[00:45]
driver locally on your server and on
[00:47]
production in a remote environment.
[00:49]
Let's begin. First, when you would need
[00:52]
cues in the cases where you don't want
[00:54]
your users to wait, for example, for
[00:56]
sending email notifications, generating
[00:58]
some reports, or processing the uploaded
[01:01]
files. In that case, you show the
[01:03]
success message here on screen and then
[01:05]
put the jobs in the background in the
[01:07]
queue. And here's a simple example. You
[01:09]
create a new task and you need to notify
[01:11]
the users that are assigned to it. And
[01:14]
then those users should get email like
[01:16]
this. But it takes a few seconds to send
[01:18]
that email. So let me show you the time
[01:20]
difference and experience difference if
[01:22]
you do it without Q and with Q in the
[01:24]
code in the controller. We have this. We
[01:26]
create the task in the store method and
[01:27]
then we send the notification to the
[01:29]
assigned users. But in this case I
[01:32]
didn't enable the queue. So what happens
[01:33]
if we open the network tab here and
[01:36]
notify three users. Let's see how much
[01:39]
it takes. Pending. Pending. A few more
[01:42]
seconds and in total 4 seconds for user
[01:46]
to wait. And imagine what happens if
[01:48]
they need to notify 10 users or 20
[01:50]
users. So this is the perfect case for
[01:52]
Laravel cues. And in case of
[01:54]
notification class in Laravel, it can be
[01:57]
queuable very easily. In fact, it is
[01:59]
queuable by default. I just disabled
[02:02]
that. So in the notification task in the
[02:04]
class, it needs to implement shoot Q and
[02:07]
use ceable. That's it. So let's try
[02:10]
again now. And by the way, those emails
[02:12]
were successfully delivered. I'm using
[02:13]
Mail Trap to test emails. So three users
[02:16]
were notified. Now let's try with a Q
[02:18]
create new task. I will use fake filler
[02:21]
chrome extension. Let's notify all four
[02:23]
users in this case. Create task and the
[02:26]
result is immediate. So success on the
[02:28]
screen for the user and then the jobs
[02:30]
for notifications go where it depends on
[02:33]
your setting of the Q driver in the
[02:36]
default. ENV4 Laravel Q connection is
[02:39]
database which means it uses database
[02:42]
driver and the default Laravel comes
[02:44]
with a few tables migrations for jobs
[02:47]
batches and failed jobs. So now if we
[02:50]
look at our database we have table jobs
[02:52]
with four items each of them
[02:54]
representing one notification email to
[02:56]
one user. They are not executed yet.
[02:58]
They are in the queue waiting for their
[03:00]
time and then the developer needs to
[03:02]
start processing the queue and you can
[03:05]
do that with command PHP artisan Q work.
[03:08]
If we launch that running then a few
[03:11]
seconds to send the emails subsequent
[03:14]
emails for some reason go faster and now
[03:16]
in the mail trap we have four new emails
[03:19]
and not only that if we leave Q work in
[03:22]
the background it becomes a background
[03:24]
process. Then look what happens if we
[03:26]
create another task. We create the task
[03:28]
and now if we go to terminal it's
[03:31]
immediately catching the new jobs. So
[03:34]
this is how the cues are supposed to
[03:36]
work on live server and we will talk
[03:38]
about that in a few minutes. There is a
[03:40]
process in the background in the queue
[03:42]
separately from the browser. So the
[03:44]
browser still shows the success result
[03:46]
quickly and then separately you need to
[03:48]
maintain the Q worker to always work and
[03:51]
always be ready for new jobs. And not
[03:54]
only notifications can be queued just
[03:56]
like that. So here's the dogs page for
[03:57]
queueing notification but also in the
[04:00]
same way you can cue mailables also by
[04:03]
adding implements shoot Q and event
[04:06]
listeners in the listener you can add
[04:08]
implements should Q for the same
[04:10]
behavior but what if you have some
[04:12]
general process like for example sending
[04:14]
a PDF invoice for that you would create
[04:16]
a custom job so here's another example
[04:19]
you create the order and you want to get
[04:21]
the invoice PDF and that PDF may take
[04:24]
quite a long time to build a few seconds
[04:26]
again. So we create a separate job with
[04:28]
artisan command php artisan make job for
[04:30]
example generate order invoice and then
[04:33]
in the handle method you do whatever you
[04:35]
want and then put that job into the
[04:37]
queue again the job implements shoot q
[04:39]
andsqable and then in the handle the job
[04:42]
is building the pdf from blade view for
[04:45]
the invoice and to put that job into the
[04:47]
queue we need to dispatch it from
[04:49]
somewhere for example from controller so
[04:51]
in the store method in addition to
[04:53]
request products and order creation we
[04:56]
have this line. So job class dispatch
[04:59]
and you may pass the parameter if you
[05:01]
need. So for example in this case we
[05:03]
have a parameter of order ID which is
[05:05]
automatically processed with PHP8
[05:08]
constructor property promotion. So you
[05:10]
don't need to create a property manually
[05:12]
and then in the handle method you use it
[05:14]
this order id and kind of a pro tip it's
[05:18]
often beneficial to re-query the data
[05:20]
from the database in the Q job because
[05:23]
by the time the job is actually executed
[05:26]
processed the data may have been changed
[05:28]
in the database and as a result if we
[05:31]
fill in the form create order look at
[05:33]
our Q work which is still running it
[05:36]
catches the job executes that and we
[05:38]
have this PDF as a result also you may
[05:40]
want to send that PDF right away. So for
[05:42]
example in the same job you may call the
[05:45]
notification. Now what happens if the
[05:47]
job fails for whatever reason some SQL
[05:49]
query error or typo in the code or
[05:52]
something like that. So the user is
[05:54]
notified about the success but then the
[05:55]
background job isn't executed
[05:57]
successfully. Let's simulate that
[05:59]
scenario and I will show you how it
[06:00]
works for database driver first locally
[06:04]
and then it's a bit different for
[06:05]
production servers with different
[06:07]
driver. We'll talk about drivers in a
[06:09]
few minutes. So for example, in the job
[06:10]
let's make a typo intentionally model
[06:13]
that doesn't exist. Now we execute the
[06:15]
same thing. Create order. Still success.
[06:17]
The order is actually created, but the
[06:20]
background job in the terminal is marked
[06:22]
as done. And this is another important
[06:25]
thing you need to understand about the
[06:27]
cues. If you change something in the
[06:29]
code, Q work doesn't automatically
[06:32]
restart and take the changes of your
[06:35]
code. It works with the Laravel version
[06:37]
at the time when that Q work was
[06:39]
started. So in our case, it actually
[06:41]
succeeded. To simulate the failed job
[06:44]
we need to restart the queue. So let's
[06:46]
stop that one and launch Q work again.
[06:49]
It will listen again for the jobs, but
[06:52]
now with this version of Laravel code.
[06:54]
So let's create another order. And now
[06:56]
what we will see is fail. This is
[06:59]
exactly what I wanted to show you. So
[07:01]
there was a job in the table of jobs.
[07:04]
But now if we refresh it's empty which
[07:06]
means the job is executed. But if it
[07:09]
fails two things happen. First you can
[07:12]
find the actual error in the log
[07:14]
laravel.log. So this is the actual error
[07:16]
like you would find any other error of
[07:18]
Laravel. But also how do you now retry
[07:21]
the job because it's not in the jobs
[07:23]
table anymore. So this is where another
[07:26]
database table comes in. Failed jobs. It
[07:29]
is logged here now with the actual error
[07:32]
exception. And if you want to retry the
[07:34]
job, for example, you fix the error. So
[07:36]
let's get it back. Then we need to
[07:38]
restart Q worker again. Q work again.
[07:40]
And this is what you need to do every
[07:42]
time you deploy new code to production.
[07:44]
You need to restart the Q worker as a
[07:46]
part of your deployment process. And now
[07:49]
our goal is to get that job from failed
[07:51]
jobs to the jobs table. And for that in
[07:54]
the separate tab, we will execute a
[07:55]
command Q retry. There are a few
[07:58]
parameters of job that you can retry.
[08:00]
For example, you can provide the ID of a
[08:03]
job. But in this case, I will just
[08:05]
choose all and it will retry the jobs.
[08:08]
So they are back from failed jobs to
[08:11]
jobs. But actually it is not here
[08:13]
because it's already automatically
[08:15]
executed, done and not failing anymore.
[08:18]
So this is how you generally manually
[08:20]
retry the jobs. But also the jobs
[08:22]
themselves have a few parameters to
[08:24]
specify how many times they should be
[08:27]
retrieded. What are the parameters for
[08:29]
max exceptions, timeout, it's so-called
[08:32]
back off, a lot of parameters that you
[08:34]
can read in the docs of Laravel. Now
[08:36]
let's talk about cues on production
[08:38]
server in live environment. This is
[08:39]
different than local database driver.
[08:42]
Here are other drivers available in
[08:44]
Laravel Q's. And the most popular one
[08:46]
and the most typical one used in Laravel
[08:49]
is Reddis. So to set up the cues on
[08:51]
production, you probably need the
[08:53]
combination of these tools. Reddis
[08:55]
here's the homepage. This tool is not
[08:57]
from Laravel or from PHP. In general
[08:59]
it's much more complicated. And we have
[09:01]
a tutorial Reddus and Laravel 101 with
[09:04]
general overview not for Q specifically
[09:07]
and I will link that in the description
[09:08]
below. But as a part of that tutorial
[09:10]
we have set up for Q connection Reddus.
[09:13]
And then that would be the overview of
[09:15]
Reddus itself. But this is where another
[09:17]
tool comes in which is Laravel Horizon.
[09:20]
This is a dashboard for Q jobs usually
[09:22]
for production server which works
[09:24]
specifically with radius. As an example
[09:26]
we had a tutorial about Q performance
[09:29]
test sending a lot of PDFs and when we
[09:31]
set up the queue with Reddis. Later in
[09:34]
the article, we could monitor how many
[09:36]
jobs were processed and the runtime and
[09:39]
the dashboard of jobs per minute and so
[09:42]
on. Solar Horizon is a must-have and a
[09:45]
very powerful tool to monitor the queue
[09:47]
jobs, but also a few commands like
[09:49]
restarting the queue should happen not
[09:52]
with manually Q termination from
[09:54]
terminal or command Q restart but a
[09:57]
specific command PHP artisan horizon
[10:00]
terminate which would take care of all
[10:02]
of the Q workers in the background and
[10:04]
that background is also provided by a
[10:06]
third tool that you need to know called
[10:08]
supervisor which is again not from
[10:10]
Laravel and not from PHP. P it's a
[10:13]
monitor or process control system to
[10:15]
restart the Q workers automatically if
[10:18]
some of them fail cuz not only job can
[10:21]
fail but also the Q worker process may
[10:23]
crash for whatever reason and in the
[10:25]
Laravel documentation there's a specific
[10:27]
section about supervisor configuration
[10:29]
how to install it and the options that
[10:32]
you may choose and if that sounds
[10:34]
complicated and it is you may install
[10:37]
all of that much easier with help of
[10:39]
tools like for example we're using
[10:41]
Laravel Forge in many of our projects.
[10:43]
It allows to just enable radius
[10:45]
connection for Q with just a few clicks.
[10:48]
Also providing the supervisor
[10:50]
automatically so you don't need to care
[10:52]
about how it works internally. Also, you
[10:54]
can do that with Laravel cloud or not
[10:56]
first-party tools like ploy for
[10:59]
deployment. Disclaimer, I'm not
[11:00]
affiliated with any forge or cloud or
[11:02]
ploy or whatever. I genuinely think that
[11:05]
setting cues up manually on VPS is just
[11:08]
not worth the time. So these are the
[11:09]
basic things about how Q works that I
[11:11]
wanted to explain in this video, but
[11:13]
there's so much more to it. The official
[11:16]
Laravel documentation is pretty long
[11:18]
with a lot of cases. And this is where
[11:20]
the main goal is when working with cues.
[11:23]
Manage them, monitor their performance
[11:26]
restart the jobs, restart the Q workers
[11:28]
scale the Q workers into multiple Q
[11:30]
workers on maybe separate server. So
[11:33]
there's a whole new world how to
[11:34]
optimize cues for really big scale. On
[11:37]
Laravel daily, we have two courses about
[11:39]
Laravel Q's just recently updated cues
[11:41]
in Laravel 12 kind of for beginners. So
[11:44]
it's this video and on top sections like
[11:46]
longunning jobs with timeouts. So, we're
[11:49]
getting deeper into the same topics that
[11:50]
I mentioned in this video. You can see
[11:52]
the chapters on the right, but also
[11:54]
there's a second course called practical
[11:56]
laral cues on live server which is a bit
[12:00]
older almost 2 years ago, but the
[12:02]
fundamentals didn't really change. Same
[12:04]
supervisor, Horizon, and Reddius. So, I
[12:07]
will link both of those courses in the
[12:08]
description below. What do you think
[12:10]
about Q's? Have I missed something
[12:11]
important or what questions do you have
[12:13]
that I could cover in separate videos?
[12:15]
That's it for this time and see you guys
[12:17]
another