The Dev Log › Programming
Laravel Queues in Production: Jobs, Retries and Horizon Without the Pain
By Jezer Niel Blanca, Full Stack Developer ·
·
6 min read
The habits that keep Laravel queues boring in production: idempotent jobs, sensible retries, timeouts, priorities, Horizon, and safe deployments.
Queues are one of those Laravel features that feel magical in development and mysterious in production. Locally, you dispatch a job, it runs, and everything is fine. Then real traffic arrives, a third-party API slows down, and suddenly you have stuck workers, duplicate emails, and a failed jobs table that keeps growing. In this post I'll walk through how I set up queues so they stay boring in production, which is exactly what you want from background work.
Why Queues Matter More Than You Think
Anything slow or unreliable should not happen inside a web request. Sending emails, generating PDFs, resizing images, syncing data with an external API, and processing webhooks all belong in the background. Moving them to a queue gives you three wins at once:
- Faster responses. The user gets a reply in milliseconds instead of waiting for an SMTP server or a remote API.
- Resilience. If a remote service is down, the job can retry later instead of throwing an error at the user.
- Control. You decide how many workers run, which work gets priority, and how failures are handled.
The trick is that queues move complexity rather than remove it. You still need to think about retries, timeouts, duplicates, and deployments. Let's go through each one.
Writing Jobs That Behave
Start with the Artisan generator so the class follows the framework conventions:
php artisan make:job SyncInvoiceToAccounting
Here is the shape I aim for in almost every job: small, explicit about retries, and safe to run more than once.
<?php
namespace App\Jobs;
use App\Models\Invoice;
use App\Services\AccountingClient;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Queue\Middleware\WithoutOverlapping;
use Illuminate\Support\Facades\Log;
use Throwable;
class SyncInvoiceToAccounting implements ShouldQueue
{
use Queueable;
public int $tries = 5;
public int $timeout = 60;
public function __construct(public Invoice $invoice) {}
/**
* @return array<int, int>
*/
public function backoff(): array
{
return [10, 60, 300];
}
/**
* @return array<int, object>
*/
public function middleware(): array
{
return [(new WithoutOverlapping($this->invoice->id))->expireAfter(120)];
}
public function handle(AccountingClient $accounting): void
{
if ($this->invoice->synced_at !== null) {
return;
}
$accounting->pushInvoice($this->invoice);
$this->invoice->update(['synced_at' => now()]);
}
public function failed(?Throwable $exception): void
{
Log::error('Invoice sync failed', [
'invoice_id' => $this->invoice->id,
'error' => $exception?->getMessage(),
]);
}
}
A few things are doing heavy lifting here:
$tries and backoff() give the remote service time to recover instead of hammering it three times in one second.
WithoutOverlapping stops two workers from syncing the same invoice at the same time.
- The early return makes the job idempotent. If it runs twice, the second run does nothing.
failed() gives you one place to alert, log, or mark the record for manual review.
Assume every job will run more than once. If running it twice would cause damage, fix the job before you ship it.
Retries, Timeouts and the retry_after Trap
The most common production bug I see with queues is a mismatch between the job timeout and the connection's retry_after value in config/queue.php. If a job is still running when retry_after expires, the queue assumes the worker died and hands the same job to another worker. Now it runs twice, in parallel.
The rule is simple: retry_after must be longer than your longest job timeout. If your slowest job has a 120 second timeout, set retry_after to something comfortably above that.
A few other retry habits that keep things calm:
- Use
retryUntil() instead of $tries when time matters more than attempts, for example "keep trying for the next 30 minutes".
- Throw exceptions for failures you want retried. Call
$this->fail() for failures that will never succeed, like a deleted record or invalid data.
- Keep jobs small. One job that syncs one invoice is much easier to retry than one job that syncs ten thousand.
Dispatch After the Transaction Commits
If you dispatch a job inside a database transaction, a fast worker can pick it up before the transaction commits and find nothing in the database. Laravel has a clean fix:
DB::transaction(function () use ($invoice) {
$invoice->markAsSent();
SyncInvoiceToAccounting::dispatch($invoice)->afterCommit();
});
You can also set after_commit to true on the connection in config/queue.php to make this the default behaviour.
Priorities and Multiple Queues
Not all work is equal. A password reset email should never wait behind a batch of report exports. I usually split work across a few named queues:
SendPasswordResetEmail::dispatch($user)->onQueue('high');
GenerateMonthlyReport::dispatch($account)->onQueue('low');
Then tell the worker to drain queues in priority order:
php artisan queue:work redis --queue=high,default,low --tries=3 --max-time=3600
The --max-time flag makes the worker exit gracefully after an hour so your process manager can start a fresh one. That keeps memory leaks from slowly building up in long-running processes.
Horizon: Queues You Can Actually See
If you are running Redis, Laravel Horizon is the single best upgrade for queue visibility. It gives you a dashboard with throughput, runtimes, failed jobs, and retries, and it lets you configure workers in code instead of in server config files.
composer require laravel/horizon
php artisan horizon:install
Your worker setup then lives in config/horizon.php, where you define supervisors per environment:
'environments' => [
'production' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['high', 'default', 'low'],
'balance' => 'auto',
'minProcesses' => 1,
'maxProcesses' => 10,
'tries' => 3,
'timeout' => 90,
],
],
],
Remember to lock down the dashboard. By default it's only available locally, and in production access is controlled by the viewHorizon gate in HorizonServiceProvider. Add only the emails or roles that should see it.
Deployments and Process Management
Workers are long-lived processes. They load your code once and keep it in memory, so after a deploy they are still running the old code until they restart. Always add a restart step to your deployment script:
php artisan horizon:terminate
# or, without Horizon
php artisan queue:restart
Both commands let workers finish their current job before exiting. Your process manager, usually Supervisor on a VPS, then starts them again with the new code. Without a process manager, a crashed worker simply stays dead, so this part is not optional.
When jobs do fail, these are the commands I keep close:
php artisan queue:failed
php artisan queue:retry all
php artisan queue:forget <id>
php artisan queue:prune-failed --hours=72
Wrapping up
Production queues are not about clever code. They are about a handful of boring habits: idempotent jobs, sensible retries, timeouts that fit inside retry_after, dispatching after commit, clear priorities, visibility with Horizon, and restarting workers on every deploy. Get those right and your background work becomes the most reliable part of your app instead of the scariest. If you are planning a product that needs reliable background processing, integrations, or heavy data work, I'd love to help you build it with my team.
Tags: Laravel, Queues, Horizon, Redis