Hangfire Jobs -Background processing with DotNet core

What's Hangfire?

Hangfire is an open-source library that allows you to kick-start the process in the background so you don't need a separate process or windows service for processing. It's also backed with persistent storage so you can pause, resume, stop and restart failed jobs at any point and it will keep track of activity in the persistent database, and everything is stored in storage which helps background jobs to survive application restart, server reboot, etc.

This library consists of three main components: Client, Storage, and Server. Below diagram will give you a high-level view of how these three components work together to complete the particular background process.

Overview diagram:

Types of background jobs

Hangfire supports several kinds of background tasks, below is the list of jobs that you can create using Hangfire. For more information about each type click here.

• Fire-And-Forget
• Delayed
• Recurring
• Continuation
• Batches
• Batch Continuation
• Background process

In this article, we will be implementing the Hangfire server in the DotNet core application and SQL as persistent storage. So let's start with actual implementations. Here we are implementing recurring jobs, but you can follow the same project for other types of jobs too.

Recurring Jobs:

• Create a new DotNet core application using Visual Studio:

Either you can create a new web application using VS or using a command prompt.

• Install Hangfire:

Hangfire is available as a NuGet package so you can install the below packages from the NuGet feed.

• Hangfire.Core
• Hangfire.SqlServer
• Hangfire.AspNetCore

dotnet add package Hangfire.Core
dotnet add package Hangfire.SqlServer
dotnet add package Hangfire.AspNetCore

Now we have all the required Hangfire packages so the next step is to prepare persistent storage, here I'm assuming that you have SQL Server installed and configured properly for use.

• Create a new Database:

As I mentioned earlier we will be using SQL Server as job storage, so connect to the local instance of the SQL server using the SSMS tool and create the database using the below command.

CREATE DATABASE [Hangfire]
GO

• Configure Hangfire Database in WebApp:

Open appsettings.json file and add a connection string.

{
  "ConnectionStrings": {
      "Hangfire": "Server=.;Database=Hangfire;Integrated Security=SSPI;"
    }
  .
  .
}

• Register services:

I have created the app using DotNet Core 6 so I don't have Startup.cs file so I will be configuring hangfire in Program.cs file, if you have DotNet 3.1 application you can use a similar configuration in Startup.cs. First, we need to import the required namespaces.

//...
using Hangfire;
using Hangfire.SqlServer;

By default DotNet core supports dependency injection and Hangfire.AspNetCore the NuGet package contains all the extension methods to register/configure Hangfire Server.

builder.Services.AddHangfire(config => config
                                .SetDataCompatibilityLevel(CompatibilityLevel.Version_170)
                                .UseSimpleAssemblyNameTypeSerializer()
                                .UseRecommendedSerializerSettings()
                                .UseSqlServerStorage(builder.Configuration.GetConnectionString("Hangfire"), new SqlServerStorageOptions
                                {
                                    CommandBatchMaxTimeout = TimeSpan.FromMinutes(5),
                                    SlidingInvisibilityTimeout = TimeSpan.FromMinutes(5),
                                    QueuePollInterval = TimeSpan.Zero,
                                    UseRecommendedIsolationLevel = true,
                                    DisableGlobalLocks = true
                                })
                            );

builder.Services.AddHangfireServer();

• Adding Dashboard UI:

Hangfire does have a super cool feature where it provides you a dashboard and you can enqueue, dequeue, re-queue, and many more operations you can perform from this UI. The following code snippet shows you how to add a dashboard UI in the web app, so you can use all features from the user-friendly UI itself and no need to remember complex commands.

app.UseEndpoints(endpoints =>
{
    endpoints.MapHangfireDashboard();
});

So finally your Program.cs file will look like the one below.

using Hangfire;
using Hangfire.SqlServer;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.

builder.Services.AddControllers();

//Configure Hangfire

builder.Services.AddHangfire(config => config
                                .SetDataCompatibilityLevel(CompatibilityLevel.Version_170)
                                .UseSimpleAssemblyNameTypeSerializer()
                                .UseRecommendedSerializerSettings()
                                .UseSqlServerStorage(builder.Configuration.GetConnectionString("Hangfire"), new SqlServerStorageOptions
                                {
                                    CommandBatchMaxTimeout = TimeSpan.FromMinutes(5),
                                    SlidingInvisibilityTimeout = TimeSpan.FromMinutes(5),
                                    QueuePollInterval = TimeSpan.Zero,
                                    UseRecommendedIsolationLevel = true,
                                    DisableGlobalLocks = true
                                })
                            );

builder.Services.AddHangfireServer();

var app = builder.Build();

app.UseHttpsRedirection();

app.UseRouting();

app.UseEndpoints(endpoints =>
{
    endpoints.MapControllers();
    endpoints.MapHangfireDashboard();
});

app.Run();

• Run the app:

Now we have everything configured to run the application so either you can press F5or run the app using the command dotnet run.

• SQL Database Schema:

In step 3 we have created separate DB for Hangfire jobs and in step 4 configured the connection in appsettings.json file, so as soon as we run the application it will initialize the SQL table schema. You can find the list of tables in the below image.

• Hangfire Dashboard:

Now our application is started, open the following URL to access the Hangfire Dashboard interface.

https://localhost:8057/hangfire

By default, the hangfire server runs on port 8057and dashboard can be served on /hangfire endpoint path but you can configure this endpoint path while registering the dashboard in Program/Startup.cs file. The following snippet will configure a custom path for accessing Dashboard.

app.UseEndpoints(endpoints =>
{
    endpoints.MapControllers();
    endpoints.MapHangfireDashboard("/dashboard");
});

Now we have everything configured to run Hangfire Server but still, we don't have any recurring job configured that's why on Recurring Jobs tab we are seeing 0 jobs on the dashboard, so let's configure one and see how we can queue and re-queue jobs from UI.

• Create a new project:

We can create new recurring jobs in the same project as well but just to keep everything isolated I have created new class library project within the same solution.

• Add UserMigration job:

Now create IUsersMigration interface and UsersMigration class which will have an actual implementation of the job.

IUsersMigration.cs

  public interface IUsersMigration
    {
        Task ExecuteAsync();
    }

UsersMigration.cs I have created this job for demo purposes so it contains basic implementation but you can implement your own logic, but I believe you got an idea about how we can create a new job.

  public class UsersMigration : IUsersMigration
    {
        public Task ExecuteAsync()
        {
            Console.WriteLine("Successfully completed!");

            return Task.CompletedTask;
        }
    }

So our entire solution structure will look like the below.

• Register newly created Job:

In Program.cs file you have to register all the jobs with cronschedule. The following snippet will register recurring jobs as Never but you can change that schedule as per your requirement.

RecurringJob.AddOrUpdate<IUsersMigration>(x => x.ExecuteAsync(), Cron.Never);

Now your final Program.cs file will look as below.

using Hangfire;
using Hangfire.SqlServer;
using Workbench.Hangfire.Jobs.Migrations.Interfaces;
using Workbench.Hangfire.Jobs.Migrations.Jobs;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.

builder.Services.AddControllers();

//Configure Hangfire

builder.Services.AddHangfire(config => config
                                .SetDataCompatibilityLevel(CompatibilityLevel.Version_170)
                                .UseSimpleAssemblyNameTypeSerializer()
                                .UseRecommendedSerializerSettings()
                                .UseSqlServerStorage(builder.Configuration.GetConnectionString("Hangfire"), new SqlServerStorageOptions
                                {
                                    CommandBatchMaxTimeout = TimeSpan.FromMinutes(5),
                                    SlidingInvisibilityTimeout = TimeSpan.FromMinutes(5),
                                    QueuePollInterval = TimeSpan.Zero,
                                    UseRecommendedIsolationLevel = true,
                                    DisableGlobalLocks = true
                                })
                            );

builder.Services.AddHangfireServer();

// Configure job's dependencies
builder.Services.AddScoped<IUsersMigration, UsersMigration>();

var app = builder.Build();

app.UseHttpsRedirection();

app.UseRouting();

app.UseEndpoints(endpoints =>
{
    endpoints.MapControllers();
    endpoints.MapHangfireDashboard("/dashboard");
});

// Register job
RecurringJob.AddOrUpdate<IUsersMigration>(x => x.ExecuteAsync(), Cron.Never);

app.Run();

• Enqueue job:

You can select as many jobs as you wanted to enqueue and click on Trigger now button, as soon as you trigger the jobs it will maintain the information in persistent storage.

To check the status you can visit the Jobs tab.

Here you can filter the list of jobs by their status so in the above image we are in Succeeded tab so all successful jobs will be listed here.

You can also check the status from SQL Server as well.

SELECT * FROM [HangFire].[Job];

GitHub Repo link.

Conclusion

Hangfire is an open-source library and is free to use for commercial use. It also provides you with a dashboard so non-technical users can use the dashboard to queue, re-queue, and delete scheduled jobs from the server. It also supports authentication to view the dashboard so no need to worry about anonymous user access. Hangfire supports SQL, Redis, and In-Memory as persistent storage so jobs can survive application restarts and even machine reboots, etc.