Setting up a service to clear Craft's job queue

In a previous post, we talked about using Redis to support a load balanced Craft site. One important point we omitted from that article was the implications to Craft's job queue handling, so today let's take a brief look at the why the queue is important, the disruption it can have when using an alternate queue component like Redis, and one very simple fix for it.

A queue you don't want to be stuck waiting on

Craft uses its internal queue to keep a list of potentially lengthy tasks, which are then performed in the background so that users of the CMS are not left waiting an inordinate amount of time for their actions to be completed. Tasks added to the queue include re-saving entries, generating image transforms, deleting template caches, updating drafts and revisions, and updating search indexes. These tasks are all important to ensure that the data presented by Craft is up to date and loads as quickly as possible, so you don't want to end up in a situation where the tasks aren't being completed and a backlog is growing.

When the queue stops moving

So what does all this have to do with Redis? Well, if you're using anything other than Craft's default queue manager you will need to have set up an alternate worker to ensure the tasks keep being completed. Without this, the search index may never be updated, or image transforms never generated.

N.B. as of Craft 3.4, there's a handy utility for viewing the state of the queue within the control panel

Getting the queue moving again

There's a variety of approaches you can take to creating a job manager service in Craft/Yii2 (the framework Craft is built on), but the one we ended up implementing was a Cron task executed every minute using Flock to prevent two instances of the task being run simultaneously in the event it takes more than one minute to process all the jobs in the queue. This is important as it prevents a nightmarish Black Friday-esque scenario where you have a queue of processes waiting to be executed to clear your queue of jobs!

To get started, SSH onto the server where your website is hosted (that's the Craft app, not the Redis server) and create a lock file - it doesn't matter where/what this is called, but I opted for the var folder and called the file cron.lock. Next, switch to the user who owns the web app and enter:

crontab -e

And add a Cron task like the following:

* * * * * /usr/bin/flock -w 0 /path/to/cron.lock /path/to/app/craft queue/run

This task is executed once per minute and will run Craft's queue/run command, with Flock abandoning the attempt until the next execution if the lock cannot be immediately acquired because the previous task is still running.

Finally, you're going to want to set the runQueueAutomatically config setting to false in config/general.php. This is effectively happening anyway, but updating the setting explicitly enables consistency and testing across environments which may not all be load balanced - as well as acting as a reminder that a separate monitor is responsible for executing the job queue.

...and that's it

Hopefully you find this post useful if for any reason Craft's default queue manager isn't keeping things moving along and you need to implement your own.