Ranjeet Karki

Posted on Sep 23, 2022 • Edited on Sep 30, 2022

Laravel chaining multiple jobs with an example

#laravel #tutorial #php #laravelqueue

In this article, I will show you how to perform Job chaining in laravel. For this example, I am uploading an excel file with thousands of rows. Laravel queue will allow importing an excel in the background without making the user wait on the page until the importing has been finished. Once the importing excel finishes, we will send an email to the user to inform him that the file was imported successfully. Job chaining helps to process multiple jobs sequentially. In our example importing an excel is one job and sending a notification email is another job. However, your case might be different. You may want to register a user and send a welcome message once they successfully registered, or you may want to process the user's orders and send an invoice to the user's email. In all these cases, you want your application to run quickly. This can be achieved with a Laravel queue that allows us to run tasks asynchronously.

For this, I am using
Laravel 9: https://laravel.com
Laravel-Excel package: https://laravel-excel.com
Mailtrap to receive the email: https://mailtrap.io

Let's start:
Part1:
Install excel package: composer require maatwebsite/excel
add the ServiceProvider in config/app.php



'providers' => [
    Maatwebsite\Excel\ExcelServiceProvider::class,
]

add the Facade in config/app.php



'aliases' => [
    'Excel' => Maatwebsite\Excel\Facades\Excel::class,
]

Part2: Create a migration file for order
My excel has orders information so I want to create an orders table and model

php artisan make:model Order -m



<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    /**
     * Run the migrations.
     *
     * @return void
     */
    public function up()
    {
        Schema::create('orders', function (Blueprint $table) {
            $table->id();
            $table->string('order');
            $table->string('order_date');
            $table->string('order_qty');
            $table->string('sales');
            $table->string('ship_model');
            $table->string('profit');
            $table->string('unit_price');
            $table->string('customer_name');
            $table->string('customer_segment');
            $table->string('product_category');
            $table->timestamps();
        });
    }

    /**
     * Reverse the migrations.
     *
     * @return void
     */
    public function down()
    {
        Schema::dropIfExists('orders');
    }
};

run php artisan queue:table to create jobs table. A jobs table hold the information about queue, payload, number of attempts etc of unprocessed jobs. Any information about failed jobs will be stored in the failed_jobs table.
and finally migrate these files by running command php artisan migrate
Part3: Let's work on .envfile
change QUEUE_CONNECTION=sync to QUEUE_CONNECTION=database
also, login to mailtrap and get your username and password
and the setting will be as below:



MAIL_MAILER=smtp
MAIL_HOST=smtp.mailtrap.io
MAIL_PORT=2525
MAIL_USERNAME=your_username_from_mailtrap
MAIL_PASSWORD=your_password_from_mailtrap
MAIL_ENCRYPTION=tls
MAIL_FROM_ADDRESS="hello@example.com"
MAIL_FROM_NAME="${APP_NAME}

Part 4: Now let's make a OrdersImport class
php artisan make:import OrdersImport --model=Order
The file can be found in app/Imports

As you can see my excel headings are not well formatted, there is space between two texts. Laravel excel package can handle this very easily with Maatwebsite\Excel\Concerns\WithHeadingRow;
Check this:Click here

So after implementing WithHeadingRow, my Excel Order Id has changed to order_id, Order Date has changed to order_date, and so on. You can check this by doing dd($row) below.



<?php

namespace App\Imports;

use App\Models\Order;
use Maatwebsite\Excel\Concerns\ToModel;
use Maatwebsite\Excel\Concerns\WithHeadingRow;

class OrdersImport implements ToModel,WithHeadingRow
{
    /**
    * @param array $row
    *
    * @return \Illuminate\Database\Eloquent\Model|null
    */
    public function model(array $row): Order
    {
        return new Order([
            'order'     => $row['order_id'],
            'order_date'    => $row['order_date'], 
            'order_qty' => $row['order_quantity'],
            'sales' => $row['sales'],
            'ship_model' => $row['ship_mode'],
            'profit' => $row['profit'],
            'unit_price' => $row['unit_price'],
            'customer_name' => $row['customer_name'],
            'customer_segment' => $row['customer_segment'],
            'product_category' => $row['product_category'],
        ]);
    }
}

Part 5: let's make a route



Route::get('upload', [UploadController::class,'index']);
Route::post('upload', [UploadController::class,'store'])->name('store');

The index method will render the view which has an upload form and store method will handle upload.
Part6: let's make UploadController with command php artisan make:controller UploadController

Part7: let's make an upload form in resources/views with the name index.blade.php



@extends('layouts.app')
@section('content')
    <div class="container">
        <div class="row justify-content-center">
            <div class="col-md-8">
                <div class="card">
                    <div class="card-header">{{ ('Upload') }}</div>
                    <div class="card-body">
                        <form action="{{ route('store') }}" method="post" enctype="multipart/form-data">
                            @csrf
                            <input type="file" name="order_file" class="form-control" required>
                            <button class="btn btn-primary" id="btn" type="submit">Upload </button>
                        </form> 
                    </div>
                </div>
            </div>
        </div>
    </div>
@endsection

Part8: Now we will make two methods in UploadController.php index method will return a view that has an upload form and the store method will have a logic to store the excel data in the database and send the user an email once the task is finished. However, we will use job classes to perform these actions.



public function index()
{
    return view('index');
}

public function store()
{
}

Part9: Now let's make a two Job class with the name ProcessUpload and SendEmail.These two files will be located in the app/Jobs directory of your project normally containing only a handle method that is invoked when the job is processed by the queue. Every job class implements the ShouldQueue interface and comes with constructor and handle() methods.
php artisan make:job ProcessUpload php artisan make:job SendEmail

ProcessUpload.php



<?php

namespace App\Jobs;

use App\Imports\OrdersImport;
use Illuminate\Bus\Queueable;
use Maatwebsite\Excel\Facades\Excel;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Contracts\Queue\ShouldBeUnique;

class ProcessUpload implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct()
    {
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        // we will write our logic to import a file 
    }
}

Part 10: Now let's work on the store method of UploadController.php. Here we will store the excel file in the storage directory and dispatch the path and email to ProcessUpload.php and SendEmail.php respectively. We will use Job Chaining, which allows us to perform multiple jobs to group together and processed them sequentially. To achieve this we have to make use of the Bus facade and call the chain method.
Now, our store method looks like this:



public function store(Request $request): string
{
    $file = $request->file('order_file')->store('temp'); 
    $path = storage_path('app'). '/' .$file;  
    $email = 'ab@gmail.com'; // or auth()->user()->email
    Bus::chain([
    new ProcessUpload($path),
    new SendEmail($email)
    ])->dispatch();

    return 'Your file is being uploaded. We will email you once it is completed';
}

Here we just passed some data to ProcessUpload.php and SendEmail.php and return a message to notify the user, remember to use use Illuminate\Support\Facades\Bus;

Now our UploadController.php looks like this



<?php

namespace App\Http\Controllers;

use App\Jobs\SendEmail;
use Illuminate\View\View;
use App\Jobs\ProcessUpload;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Bus;

class UploadController extends Controller
{
    public function index(): View
    {
        return view('index');
    }

    public function store(Request $request): string
    {
        $file = $request->file('order_file')->store('temp');
        $path = storage_path('app') . '/' . $file;
        $email = 'ab@gmail.com'; //auth()->user()->email
        Bus::chain([
            new ProcessUpload($path),
            new SendEmail($email)
        ])->dispatch();

        return 'Your file is being uploaded. We will email you once it is completed';
    }
}

Note:

Jobs can also be arranged in chains or batches. Both allow multiple jobs to be grouped together. The main difference between a batch and a chain is that jobs in a batch are processed simultaneously, Whereas jobs in a chain are processed sequentially. By making use of the chain method if one job fails to process the whole sequence will fail to process, which means if importing an excel file fails, sending a notification email will never be proceeded. So based on the type of work you want to perform, either you can use batch() or chain(). One advantage of using batch is that it helps to track the progress of your job. All the information like the total number of jobs, number of pending jobs, number of failed jobs etc will be stored in a job_batches table

Now let's work on ProcessUpload.php and SendEmail.php
first ProcessUpload.php



<?php

namespace App\Jobs;

use App\Imports\OrdersImport;
use Illuminate\Bus\Queueable;
use Maatwebsite\Excel\Facades\Excel;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Contracts\Queue\ShouldBeUnique;

class ProcessUpload implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public string $data;
    public function __construct($data)
    {
        $this->data  = $data;
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle(): void
    {
        Excel::import(new OrdersImport, $this->data);    
    }
}

As you can see in the handle method, we have received the file name as $this->data,and we used the Excel facade provided by the package to send those excel data to OrdersImport, and the rest OrdersImport will handle. OrdersImport will insert data into the database.

Remember to use Maatwebsite\Excel\Facades\Excel; as above

Part11: Now before we work on SendEmail.php, let's make Mailables with the name NotificationEmail.php and
These classes will be stored in the app/Mail directory.

php artisan make:mail NotificationEmail

This is a very simple class that looks like this



<?php

namespace App\Mail;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Mail\Mailable;
use Illuminate\Queue\SerializesModels;

class NotificationEmail extends Mailable
{
    use Queueable, SerializesModels;

    /**
     * Create a new message instance.
     *
     * @return void
     */
    public function __construct()
    {
        //
    }

    /**
     * Build the message.
     *
     * @return $this
     */
    public function build()
    {
        return $this->view('notification.mail');
    }
}

Now let's make mail.blade.php inside resources/views/notification folder

mail.blade.php



<!DOCTYPE html>
<html>
<head>
    <title>Success Email</title>
</head>
<body>
<p>Congratulation! Your file was successfully imported.😃</p>
</body>
</html>

The message is simple. We just want to send the above message to user email when file imports finish .The final part is to work on SendEmail.php

Let's work on SendEmail.php



<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use App\Mail\NotificationEmail;
use Illuminate\Support\Facades\Mail;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Contracts\Queue\ShouldBeUnique;

class SendEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public string $email;
    public function __construct($email)
    {
        $this->email = $email; 
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle(): void
    {
        Mail::to($this->email)->send(new NotificationEmail());
    }
}

To send a message, we use Mail facade. We used the Mail facade and specified the recipient's email. Finally, we are sending mail view to user which is specified in NotificationEmail class

Now final part is to run the queue and upload the excel file.
To run the queue worker php artisan queue:work inside your project directory. A queue worker is a regular process that runs in the background and begin processing the unprocessed job.
Now. visit /upload and upload a excel file

Now you should see this in your terminal

This means both tasks, importing an excel file and sending an email was successful.
Now let's check mailtrap for the email, and there should be the email sent to you.

Delete files from Storage

Did you notice we have stored all the excel files in the storage folder of our app? There is no reason to keep these files there forever as we no longer need them once we import all their data into the database.Therefore, we have to delete it from the storage. For this let's make another job with the name DeleteFile.
php artisan make:job DeleteFile
Now, add this in UploadController within the store method as below:



 public function store(Request $request): string
 {
      $file = $request->file('order_file')->store('temp');
      $path = storage_path('app') . '/' . $file;
      $email = 'ab@gmail.com'; //auth()->user()->email
      Bus::chain([
          new ProcessUpload($path),
          new SendEmail($email),
          new DeleteFile($file)// new class added
      ])->dispatch();

      return 'Your file is being uploaded. We will email you once it is completed';
  }

We passed $file in DeleteFile class, if you dd($file), you will get something like thistemp/M3aj6Ee29CdgmrW9USwUezmEHpBmlV0DkXP8P0ce.xlsx where temp is the folder inside storage/app directory that store all the uploaded excel files.Now, we have to write the logic in DeleteFile.php in handle method to delete the file from storage.
So, our DeleteFile.php looks like this:



<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Storage;

class DeleteFile implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public string $file;
    public function __construct($file)
    {
        $this->file = $file;
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        unlink(storage_path('app/'. $this->file));
    }
}

Restart queue: php artisan queue:work

The unlink() function is an inbuilt function in PHP which is used to delete files

Dealing With Failed Jobs

Sometime things do not work as expected. There may be a chance that jobs fail. The good thing is that Laravel provides a way to retry failed jobs. Laravel includes a convenient way to specify the maximum number of times a job should be attempted.
Check this
In a job class, we can declare the public $tries property and specify the number of times you want Laravel to retry the process once it fails. You can also calculate the number of seconds to wait before retrying the job. This can be defined in the $backoff array. Let's take the example of SendEmail.php and let's throw the plain exception within the handle method and also declare $tries and $backoffproperty.
SendEmail.php



<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use App\Mail\NotificationEmail;
use Illuminate\Support\Facades\Mail;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

class SendEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public string $email;
    public $tries = 3;
    public $backoff = [10, 30, 60];

    public function __construct($email)
    {
        $this->email = $email;
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle(): void
    {
        throw new Exception();
       // Mail::to($this->email)->send(new NotificationEmail());
    }
}

It's a good idea to include $tries and $backoff property in every job class.In the above example, I have commented on the Mail sending feature and declared a plain exception just to test the retry features of Laravel.Now restart the queue: php artisan queue:work and upload the file.
Now you should see SendEmail job was processed three times because we have mentioned public $tries = 3 and
when the first job failed, Laravel tried after 10 seconds, when the second job failed, Laravel tried in 30 seconds, and when the third job failed Laravel tried in 60 seconds. If it still fails after the third attempt, failed jobs will be stored in the failed_jobs table.

I hope this was helpful
Thank you :)