Laravel Queue & Jobs: Best Practices cho Production

Giới thiệu

Queues là critical infrastructure cho production Laravel applications. Email, notifications, image processing, API calls - tất cả nên được push vào queue để không block user requests.

Bài viết này cover từ basics đến advanced patterns cho production-ready queue implementation.

Khi nào cần Queue?

✅ Nên dùng Queue:

1. Email sending - SMTP có thể chậm/fail
2. File processing - Image resize, PDF generation
3. Third-party API calls - Payment, shipping, notifications
4. Heavy computations - Reports, analytics
5. Scheduled tasks - Cleanup, sync

❌ Không cần Queue:

1. Quick database writes - < 100ms
2. User cần kết quả ngay - Real-time requirement
3. Simple operations - No external dependencies

Queue Drivers

Development: sync

QUEUE_CONNECTION=sync

Jobs chạy synchronously - dễ debug nhưng không phản ánh production behavior.

Production: redis

QUEUE_CONNECTION=redis
REDIS_HOST=127.0.0.1
REDIS_PORT=6379

Fast, reliable, supports nhiều features.

Alternative: database

QUEUE_CONNECTION=database

php artisan queue:table
php artisan migrate

Đơn giản, không cần Redis, nhưng performance thấp hơn.

Creating Jobs

Basic Job

<?php
// app/Jobs/SendWelcomeEmail.php

namespace App\Jobs;

use App\Models\User;
use App\Mail\WelcomeEmail;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Mail;

class SendWelcomeEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function __construct(
        public User $user
    ) {}

    public function handle(): void
    {
        Mail::to($this->user->email)
            ->send(new WelcomeEmail($this->user));
    }
}

Dispatching Jobs

// Basic dispatch
SendWelcomeEmail::dispatch($user);

// Với delay
SendWelcomeEmail::dispatch($user)
    ->delay(now()->addMinutes(10));

// Với queue cụ thể
SendWelcomeEmail::dispatch($user)
    ->onQueue('emails');

// Với connection cụ thể
SendWelcomeEmail::dispatch($user)
    ->onConnection('redis');

// Chain jobs
Bus::chain([
    new ProcessPodcast($podcast),
    new OptimizePodcast($podcast),
    new PublishPodcast($podcast),
])->dispatch();

Error Handling

Retry Configuration

class ProcessOrder implements ShouldQueue
{
    // Số lần retry
    public int $tries = 3;
    
    // Hoặc timeout thay vì số lần
    public int $maxExceptions = 3;
    
    // Backoff giữa các lần retry
    public int $backoff = 60; // seconds
    
    // Progressive backoff
    public function backoff(): array
    {
        return [60, 300, 600]; // 1min, 5min, 10min
    }
    
    // Timeout cho job
    public int $timeout = 120;
}

Handling Failures

class ProcessOrder implements ShouldQueue
{
    public function handle(): void
    {
        // Process order...
    }
    
    public function failed(\Throwable $exception): void
    {
        // Notify admin
        Log::error('Order processing failed', [
            'order_id' => $this->order->id,
            'error' => $exception->getMessage(),
        ]);
        
        // Send alert
        Notification::send(
            User::admins()->get(),
            new JobFailedNotification($this->order, $exception)
        );
    }
}

Rate Limiting

use Illuminate\Support\Facades\RateLimiter;

class CallExternalApi implements ShouldQueue
{
    public function handle(): void
    {
        // Release job nếu rate limited
        if (RateLimiter::tooManyAttempts('external-api', 100)) {
            return $this->release(60); // Retry sau 60s
        }
        
        RateLimiter::hit('external-api');
        
        // Call API...
    }
}

Unique Jobs

use Illuminate\Contracts\Queue\ShouldBeUnique;

class UpdateSearchIndex implements ShouldQueue, ShouldBeUnique
{
    public function __construct(
        public Product $product
    ) {}
    
    // Unique key - không dispatch nếu job cùng key đang pending
    public function uniqueId(): string
    {
        return $this->product->id;
    }
    
    // Thời gian giữ unique lock
    public int $uniqueFor = 3600;
}

Queue Workers

Development

php artisan queue:work

Production với Supervisor

# /etc/supervisor/conf.d/laravel-worker.conf

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/app/artisan queue:work redis --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=8
redirect_stderr=true
stdout_logfile=/var/www/app/storage/logs/worker.log
stopwaitsecs=3600

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start laravel-worker:*

Worker Options quan trọng

php artisan queue:work \
    --queue=high,default,low \  # Priority queues
    --sleep=3 \                  # Sleep khi không có jobs
    --tries=3 \                  # Số lần retry
    --max-time=3600 \            # Restart worker sau 1 giờ
    --max-jobs=1000 \            # Restart sau 1000 jobs
    --memory=128                 # Restart nếu vượt 128MB

Laravel Horizon

Installation

composer require laravel/horizon
php artisan horizon:install
php artisan migrate

Configuration

// config/horizon.php
'environments' => [
    'production' => [
        'supervisor-1' => [
            'connection' => 'redis',
            'queue' => ['high', 'default', 'low'],
            'balance' => 'auto',  // Auto-scaling
            'processes' => 10,
            'tries' => 3,
            'timeout' => 300,
        ],
    ],
],

Running Horizon

# Development
php artisan horizon

# Production với Supervisor
[program:horizon]
process_name=%(program_name)s
command=php /var/www/app/artisan horizon
autostart=true
autorestart=true
user=www-data
redirect_stderr=true
stdout_logfile=/var/www/app/storage/logs/horizon.log
stopwaitsecs=3600

Dashboard Access Control

// app/Providers/HorizonServiceProvider.php
protected function gate(): void
{
    Gate::define('viewHorizon', function ($user) {
        return in_array($user->email, [
            'admin@example.com',
        ]);
    });
}

Advanced Patterns

Job Batching

use Illuminate\Bus\Batch;
use Illuminate\Support\Facades\Bus;

$batch = Bus::batch([
    new ProcessPodcast($podcast1),
    new ProcessPodcast($podcast2),
    new ProcessPodcast($podcast3),
])
->then(function (Batch $batch) {
    // Tất cả jobs hoàn thành
    Log::info('All podcasts processed');
})
->catch(function (Batch $batch, \Throwable $e) {
    // Job đầu tiên fail
    Log::error('Batch failed', ['error' => $e->getMessage()]);
})
->finally(function (Batch $batch) {
    // Batch hoàn thành (có thể có failures)
})
->dispatch();

// Check batch status
$batch = Bus::findBatch($batchId);
echo $batch->progress(); // 0-100

Job Middleware

// app/Jobs/Middleware/RateLimited.php
class RateLimited
{
    public function handle($job, $next)
    {
        if (RateLimiter::tooManyAttempts('api-calls', 60)) {
            return $job->release(30);
        }
        
        RateLimiter::hit('api-calls');
        
        $next($job);
    }
}

// Sử dụng trong Job
public function middleware(): array
{
    return [new RateLimited];
}

Queue Events

// app/Providers/AppServiceProvider.php
use Illuminate\Queue\Events\JobFailed;
use Illuminate\Queue\Events\JobProcessed;
use Illuminate\Queue\Events\JobProcessing;

public function boot(): void
{
    Queue::failing(function (JobFailed $event) {
        // Log, notify...
    });
    
    Queue::before(function (JobProcessing $event) {
        // Before processing...
    });
    
    Queue::after(function (JobProcessed $event) {
        // After processing...
    });
}

Monitoring & Alerting

Failed Jobs Table

php artisan queue:failed-table
php artisan migrate

// Retry failed jobs
php artisan queue:retry all

// Xóa failed jobs
php artisan queue:flush

Custom Monitoring

// Scheduled command để check queue health
// app/Console/Commands/MonitorQueues.php

public function handle(): void
{
    $pending = DB::table('jobs')->count();
    $failed = DB::table('failed_jobs')
        ->where('failed_at', '>', now()->subHour())
        ->count();
    
    if ($pending > 1000) {
        Notification::send(
            User::admins()->get(),
            new QueueBacklogAlert($pending)
        );
    }
    
    if ($failed > 10) {
        Notification::send(
            User::admins()->get(),
            new HighFailureRateAlert($failed)
        );
    }
}

Production Checklist

Configuration

• Use Redis driver
• Configure Horizon / Supervisor
• Set proper timeouts and retries
• Configure failed jobs table

Error Handling

• Implement failed() method
• Set up failure notifications
• Configure rate limiting where needed

Monitoring

• Install Horizon dashboard
• Set up alerting for failures
• Monitor queue backlog

Scaling

• Use queue priorities (high, default, low)
• Configure auto-scaling với Horizon
• Separate workers for different queue types

Common Mistakes

❌ Large payloads trong Job

// ❌ Bad: Serialize cả object
class ProcessOrder implements ShouldQueue
{
    public function __construct(
        public Collection $allProducts // Có thể rất lớn
    ) {}
}

// ✅ Good: Chỉ pass ID
class ProcessOrder implements ShouldQueue
{
    public function __construct(
        public int $orderId
    ) {}
    
    public function handle(): void
    {
        $order = Order::find($this->orderId);
        // Process...
    }
}

❌ Không handle model deletion

// ❌ Job fail nếu user bị xóa trước khi process
class SendEmail implements ShouldQueue
{
    public function __construct(public User $user) {}
}

// ✅ Handle gracefully
class SendEmail implements ShouldQueue
{
    public bool $deleteWhenMissingModels = true;
}

Kết luận

Laravel Queues là powerful tool cho production applications:

1. Chọn đúng driver - Redis cho production
2. Handle failures properly - Retry, notifications
3. Monitor actively - Horizon, custom alerts
4. Scale appropriately - Workers, priorities

Golden rule: Nếu operation có thể fail hoặc mất > 1 giây - đưa vào queue.