[0:00] Hello guys, today let's talk about
[0:01] Laravel Q's. I will try to take this
[0:03] very complex topic and explain the main
[0:05] things that you need to know about Q's
[0:07] in roughly 10 to 15 minutes. And this is
[0:09] kind of a challenge I decided for
[0:10] myself. Take some complex topic in
[0:13] Laravel and try to explain it in 10 to
[0:15] 15 minutes. So you will see more of
[0:16] these videos in the future. How did I go
[0:18] with this simplification? Shoot in the
[0:20] comments below. So today we'll talk
[0:22] about how to put the email notification
[0:24] in a queue with this example and I'll
[0:26] show you how different the experience is
[0:28] with and without Q's. Then the second
[0:30] example would be how to create a Q job
[0:33] to build the PDF invoice for the order
[0:35] and the link to code repository for both
[0:37] examples will be in the description
[0:38] below. Also we'll talk about how to deal
[0:40] with Q job failures, how to generally
[0:43] run the Q and what should be the Q
[0:45] driver locally on your server and on
[0:47] production in a remote environment.
[0:49] Let's begin. First, when you would need
[0:52] cues in the cases where you don't want
[0:54] your users to wait, for example, for
[0:56] sending email notifications, generating
[0:58] some reports, or processing the uploaded
[1:01] files. In that case, you show the
[1:03] success message here on screen and then
[1:05] put the jobs in the background in the
[1:07] queue. And here's a simple example. You
[1:09] create a new task and you need to notify
[1:11] the users that are assigned to it. And
[1:14] then those users should get email like
[1:16] this. But it takes a few seconds to send
[1:18] that email. So let me show you the time
[1:20] difference and experience difference if
[1:22] you do it without Q and with Q in the
[1:24] code in the controller. We have this. We
[1:26] create the task in the store method and
[1:27] then we send the notification to the
[1:29] assigned users. But in this case I
[1:32] didn't enable the queue. So what happens
[1:33] if we open the network tab here and
[1:36] notify three users. Let's see how much
[1:39] it takes. Pending. Pending. A few more
[1:42] seconds and in total 4 seconds for user
[1:46] to wait. And imagine what happens if
[1:48] they need to notify 10 users or 20
[1:50] users. So this is the perfect case for
[1:52] Laravel cues. And in case of
[1:54] notification class in Laravel, it can be
[1:57] queuable very easily. In fact, it is
[1:59] queuable by default. I just disabled
[2:02] that. So in the notification task in the
[2:04] class, it needs to implement shoot Q and
[2:07] use ceable. That's it. So let's try
[2:10] again now. And by the way, those emails
[2:12] were successfully delivered. I'm using
[2:13] Mail Trap to test emails. So three users
[2:16] were notified. Now let's try with a Q
[2:18] create new task. I will use fake filler
[2:21] chrome extension. Let's notify all four
[2:23] users in this case. Create task and the
[2:26] result is immediate. So success on the
[2:28] screen for the user and then the jobs
[2:30] for notifications go where it depends on
[2:33] your setting of the Q driver in the
[2:36] default. ENV4 Laravel Q connection is
[2:39] database which means it uses database
[2:42] driver and the default Laravel comes
[2:44] with a few tables migrations for jobs
[2:47] batches and failed jobs. So now if we
[2:50] look at our database we have table jobs
[2:52] with four items each of them
[2:54] representing one notification email to
[2:56] one user. They are not executed yet.
[2:58] They are in the queue waiting for their
[3:00] time and then the developer needs to
[3:02] start processing the queue and you can
[3:05] do that with command PHP artisan Q work.
[3:08] If we launch that running then a few
[3:11] seconds to send the emails subsequent
[3:14] emails for some reason go faster and now
[3:16] in the mail trap we have four new emails
[3:19] and not only that if we leave Q work in
[3:22] the background it becomes a background
[3:24] process. Then look what happens if we
[3:26] create another task. We create the task
[3:28] and now if we go to terminal it's
[3:31] immediately catching the new jobs. So
[3:34] this is how the cues are supposed to
[3:36] work on live server and we will talk
[3:38] about that in a few minutes. There is a
[3:40] process in the background in the queue
[3:42] separately from the browser. So the
[3:44] browser still shows the success result
[3:46] quickly and then separately you need to
[3:48] maintain the Q worker to always work and
[3:51] always be ready for new jobs. And not
[3:54] only notifications can be queued just
[3:56] like that. So here's the dogs page for
[3:57] queueing notification but also in the
[4:00] same way you can cue mailables also by
[4:03] adding implements shoot Q and event
[4:06] listeners in the listener you can add
[4:08] implements should Q for the same
[4:10] behavior but what if you have some
[4:12] general process like for example sending
[4:14] a PDF invoice for that you would create
[4:16] a custom job so here's another example
[4:19] you create the order and you want to get
[4:21] the invoice PDF and that PDF may take
[4:24] quite a long time to build a few seconds
[4:26] again. So we create a separate job with
[4:28] artisan command php artisan make job for
[4:30] example generate order invoice and then
[4:33] in the handle method you do whatever you
[4:35] want and then put that job into the
[4:37] queue again the job implements shoot q
[4:39] andsqable and then in the handle the job
[4:42] is building the pdf from blade view for
[4:45] the invoice and to put that job into the
[4:47] queue we need to dispatch it from
[4:49] somewhere for example from controller so
[4:51] in the store method in addition to
[4:53] request products and order creation we
[4:56] have this line. So job class dispatch
[4:59] and you may pass the parameter if you
[5:01] need. So for example in this case we
[5:03] have a parameter of order ID which is
[5:05] automatically processed with PHP8
[5:08] constructor property promotion. So you
[5:10] don't need to create a property manually
[5:12] and then in the handle method you use it
[5:14] this order id and kind of a pro tip it's
[5:18] often beneficial to re-query the data
[5:20] from the database in the Q job because
[5:23] by the time the job is actually executed
[5:26] processed the data may have been changed
[5:28] in the database and as a result if we
[5:31] fill in the form create order look at
[5:33] our Q work which is still running it
[5:36] catches the job executes that and we
[5:38] have this PDF as a result also you may
[5:40] want to send that PDF right away. So for
[5:42] example in the same job you may call the
[5:45] notification. Now what happens if the
[5:47] job fails for whatever reason some SQL
[5:49] query error or typo in the code or
[5:52] something like that. So the user is
[5:54] notified about the success but then the
[5:55] background job isn't executed
[5:57] successfully. Let's simulate that
[5:59] scenario and I will show you how it
[6:00] works for database driver first locally
[6:04] and then it's a bit different for
[6:05] production servers with different
[6:07] driver. We'll talk about drivers in a
[6:09] few minutes. So for example, in the job
[6:10] let's make a typo intentionally model
[6:13] that doesn't exist. Now we execute the
[6:15] same thing. Create order. Still success.
[6:17] The order is actually created, but the
[6:20] background job in the terminal is marked
[6:22] as done. And this is another important
[6:25] thing you need to understand about the
[6:27] cues. If you change something in the
[6:29] code, Q work doesn't automatically
[6:32] restart and take the changes of your
[6:35] code. It works with the Laravel version
[6:37] at the time when that Q work was
[6:39] started. So in our case, it actually
[6:41] succeeded. To simulate the failed job
[6:44] we need to restart the queue. So let's
[6:46] stop that one and launch Q work again.
[6:49] It will listen again for the jobs, but
[6:52] now with this version of Laravel code.
[6:54] So let's create another order. And now
[6:56] what we will see is fail. This is
[6:59] exactly what I wanted to show you. So
[7:01] there was a job in the table of jobs.
[7:04] But now if we refresh it's empty which
[7:06] means the job is executed. But if it
[7:09] fails two things happen. First you can
[7:12] find the actual error in the log
[7:14] laravel.log. So this is the actual error
[7:16] like you would find any other error of
[7:18] Laravel. But also how do you now retry
[7:21] the job because it's not in the jobs
[7:23] table anymore. So this is where another
[7:26] database table comes in. Failed jobs. It
[7:29] is logged here now with the actual error
[7:32] exception. And if you want to retry the
[7:34] job, for example, you fix the error. So
[7:36] let's get it back. Then we need to
[7:38] restart Q worker again. Q work again.
[7:40] And this is what you need to do every
[7:42] time you deploy new code to production.
[7:44] You need to restart the Q worker as a
[7:46] part of your deployment process. And now
[7:49] our goal is to get that job from failed
[7:51] jobs to the jobs table. And for that in
[7:54] the separate tab, we will execute a
[7:55] command Q retry. There are a few
[7:58] parameters of job that you can retry.
[8:00] For example, you can provide the ID of a
[8:03] job. But in this case, I will just
[8:05] choose all and it will retry the jobs.
[8:08] So they are back from failed jobs to
[8:11] jobs. But actually it is not here
[8:13] because it's already automatically
[8:15] executed, done and not failing anymore.
[8:18] So this is how you generally manually
[8:20] retry the jobs. But also the jobs
[8:22] themselves have a few parameters to
[8:24] specify how many times they should be
[8:27] retrieded. What are the parameters for
[8:29] max exceptions, timeout, it's so-called
[8:32] back off, a lot of parameters that you
[8:34] can read in the docs of Laravel. Now
[8:36] let's talk about cues on production
[8:38] server in live environment. This is
[8:39] different than local database driver.
[8:42] Here are other drivers available in
[8:44] Laravel Q's. And the most popular one
[8:46] and the most typical one used in Laravel
[8:49] is Reddis. So to set up the cues on
[8:51] production, you probably need the
[8:53] combination of these tools. Reddis
[8:55] here's the homepage. This tool is not
[8:57] from Laravel or from PHP. In general
[8:59] it's much more complicated. And we have
[9:01] a tutorial Reddus and Laravel 101 with
[9:04] general overview not for Q specifically
[9:07] and I will link that in the description
[9:08] below. But as a part of that tutorial
[9:10] we have set up for Q connection Reddus.
[9:13] And then that would be the overview of
[9:15] Reddus itself. But this is where another
[9:17] tool comes in which is Laravel Horizon.
[9:20] This is a dashboard for Q jobs usually
[9:22] for production server which works
[9:24] specifically with radius. As an example
[9:26] we had a tutorial about Q performance
[9:29] test sending a lot of PDFs and when we
[9:31] set up the queue with Reddis. Later in
[9:34] the article, we could monitor how many
[9:36] jobs were processed and the runtime and
[9:39] the dashboard of jobs per minute and so
[9:42] on. Solar Horizon is a must-have and a
[9:45] very powerful tool to monitor the queue
[9:47] jobs, but also a few commands like
[9:49] restarting the queue should happen not
[9:52] with manually Q termination from
[9:54] terminal or command Q restart but a
[9:57] specific command PHP artisan horizon
[10:00] terminate which would take care of all
[10:02] of the Q workers in the background and
[10:04] that background is also provided by a
[10:06] third tool that you need to know called
[10:08] supervisor which is again not from
[10:10] Laravel and not from PHP. P it's a
[10:13] monitor or process control system to
[10:15] restart the Q workers automatically if
[10:18] some of them fail cuz not only job can
[10:21] fail but also the Q worker process may
[10:23] crash for whatever reason and in the
[10:25] Laravel documentation there's a specific
[10:27] section about supervisor configuration
[10:29] how to install it and the options that
[10:32] you may choose and if that sounds
[10:34] complicated and it is you may install
[10:37] all of that much easier with help of
[10:39] tools like for example we're using
[10:41] Laravel Forge in many of our projects.
[10:43] It allows to just enable radius
[10:45] connection for Q with just a few clicks.
[10:48] Also providing the supervisor
[10:50] automatically so you don't need to care
[10:52] about how it works internally. Also, you
[10:54] can do that with Laravel cloud or not
[10:56] first-party tools like ploy for
[10:59] deployment. Disclaimer, I'm not
[11:00] affiliated with any forge or cloud or
[11:02] ploy or whatever. I genuinely think that
[11:05] setting cues up manually on VPS is just
[11:08] not worth the time. So these are the
[11:09] basic things about how Q works that I
[11:11] wanted to explain in this video, but
[11:13] there's so much more to it. The official
[11:16] Laravel documentation is pretty long
[11:18] with a lot of cases. And this is where
[11:20] the main goal is when working with cues.
[11:23] Manage them, monitor their performance
[11:26] restart the jobs, restart the Q workers
[11:28] scale the Q workers into multiple Q
[11:30] workers on maybe separate server. So
[11:33] there's a whole new world how to
[11:34] optimize cues for really big scale. On
[11:37] Laravel daily, we have two courses about
[11:39] Laravel Q's just recently updated cues
[11:41] in Laravel 12 kind of for beginners. So
[11:44] it's this video and on top sections like
[11:46] longunning jobs with timeouts. So, we're
[11:49] getting deeper into the same topics that
[11:50] I mentioned in this video. You can see
[11:52] the chapters on the right, but also
[11:54] there's a second course called practical
[11:56] laral cues on live server which is a bit
[12:00] older almost 2 years ago, but the
[12:02] fundamentals didn't really change. Same
[12:04] supervisor, Horizon, and Reddius. So, I
[12:07] will link both of those courses in the
[12:08] description below. What do you think
[12:10] about Q's? Have I missed something
[12:11] important or what questions do you have
[12:13] that I could cover in separate videos?
[12:15] That's it for this time and see you guys
[12:17] another