What every ASP.NET Core Web API project needs - Part 1 - Serilog

In a series of articles, I'm going to show the implementation of an architecture that is suitable for a thin Web API project or Web API that fits in a microservices architecture. In the first few articles, I'm going to introduce several useful libraries.

Let's get started with logging. Logging is just essential for debugging, troubleshooting and monitoring. A good logging system makes life much easier.

Why Serilog? It is easy to set up, has a clean API, and is portable between recent .NET platforms. The big difference between Serilog and the other frameworks is that it is designed to do structured logging out of the box. Another thing I really like about Serilog is that it can be configured via the appsetting.json file alongside configuring through code. Changing logging configuration without touching the codebase is really helpful, especially in the production environment.

Let's start with creating a new project.

Step 1 - New project

Create a new ASP.NET Core 5.0 API project.

Step 2 - Install package

Install Serilog.AspNetCore nuget package.

Step 3 - Add UseSerilog extension method

Open Program.cs file and modify CreateHostBuilder method:

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup<Startup>();
        })
        .UseSerilog((hostingContext, loggerConfiguration) =>
            loggerConfiguration.ReadFrom.Configuration(hostingContext.Configuration));

UseSerilog sets Serilog as the logging provider. We are going to config Serilog via the appsettings.json file.

Step 4 - Remove default configuration

Open appsettings.json and appsettings.Development.json file and get rid of the logging section:

"Logging": {
  "LogLevel": {
    "Default": "Information",
    "Microsoft": "Warning",
    "Microsoft.Hosting.Lifetime": "Information"
  }
}

Step 5: Add Serilog configuration

Add Serilog configuration section to appsettings.Development.json file:

"Serilog": {
  "MinimumLevel": {
    "Default": "Debug",
    "Override": {
      "Microsoft": "Information",
      "System": "Information"
    },
    "Using": [ ],
  },
  "WriteTo": [
    { }
  ]
}

Serilog.AspNetCore nuget package has dependency on Serilog.Settings.Configuration nuget package and it is a Serilog settings provider that reads from Microsoft.Extensions.Configuration sources. The above configuration is equivalent to this:

Log.Logger = new LoggerConfiguration()
    .MinimumLevel.Debug()
    .MinimumLevel.Override("Microsoft", LogEventLevel.Information)
    .MinimumLevel.Override("System", LogEventLevel.Information)
    .Enrich.FromLogContext()
    .CreateLogger();

Step 6 - Installing Sinks

Serilog uses sinks to write log events to storage for example database, file, etc. One of the most popular sinks for debugging environment is the Console sink.

• Install Serilog.Sinks.Consolenuget package
• Add following configuration:

"Serilog": {
  "MinimumLevel": {
    "Default": "Debug",
    "Override": {
      "Microsoft": "Information",
      "System": "Information"
    }
  },
  "Using": [ "Serilog.Sinks.Console" ],
  "WriteTo": [
    { "Name": "Console" }
  ]
}

After installing the sink

• In the Using section add the sink's nuget package name "Using": [ "Serilog.Sinks.Console" ]
• In the WriteTo section add sink name and arguments "WriteTo":[ { "Name": "Console" } ]

Now we want to use SQL Server sink for other environments:

• Install Serilog.Sinks.MSSqlServer sink
• Copy Serilog setting form appsettings.Development.json to appsettings.json file

"Serilog": {
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft": "Error",
      "System": "Error"
    },
    "Using": [ "Serilog.Sinks.MSSqlServer" ]
  },
  "WriteTo": [
    {
      "Name": "MSSqlServer",
      "Args": {
        "connectionString": "ConnectionString",
        "tableName": "Logs",
        "autoCreateSqlTable": true
      }
    }
  ]
}

Step 7 - How to configure a sink

Well, configuring a sink via appsettings.json could be harder than configuring through the code, and for each sink, you might not be able to find a JSON configuration sample. Normally each sink accepts several parameters to configure the sink. For instance, the Console sink accepts the below parameters:

Each one of these parameters can be configured through JSON:

"WriteTo": [
  {
    "Name": "Console",
    "Args": {
      "restrictedToMinimumLevel": "Verbose",
      "outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj} <s:{SourceContext}>{NewLine}{Exception}",
      "theme": "Serilog.Sinks.SystemConsole.Themes.AnsiConsoleTheme::Code, Serilog.Sinks.Console"
    }
  }
]

To see complete SQL Server sink JSON configuration check out this.

Step 8 - Enrichers

Log events can be enriched with properties in various ways. You can add additional data by enrichers. For instance, in the production environment, we want to add the IP of the client to the log events.

• Install Serilog.Enrichers.ClientInfo package
• Add enriched package name to Using section
• Add Enrich section with WithClientIp value (enriched name normally starts with With word)

"Using": [ "Serilog.Sinks.MSSqlServer", "Serilog.Enrichers.ClientInfo" ],
"Enrich": [ "WithClientIp" ]

All events written through log will carry a property ClientIp with the IP of the client. Check out the list of available enrichers here.

Step 9 - Filters

By using filters you can exclude or include some log events.

• Install Serilog.Expressions nuget package
• Add the "Filter" section to Serilog settings

"Filter": [
  {
    "Name": "ByExcluding",
    "Args": {
      "expression": "RequestPath like '%swagger%'"
    }
  }
]

All log events that contain swagger will be excluded.

To see all possible configurations check out Serilog.Settings.Configuration Github repository.

Step 10 - HTTP requests loging

Moreover, you can log the HTTP requests.

• In Startup.cs file, add the middleware with UseSerilogRequestLogging():

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    ...
    app.UseSerilogRequestLogging();

• In MinimumLevel.Override section add "Microsoft.AspNetCore": "Warning":

"MinimumLevel": {
  "Default": "Information",
  "Override": {
    "Microsoft": "Error",
    "Microsoft.AspNetCore": "Warning",
    "System": "Error"
  },

Step 11 - Overridng configuration in docker

Last but not least here I want to mention is that you can override the Serilog setting by Docker environment variable. Consider the following configuration:

"Serilog": {
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft": "Error",
      "System": "Error"
    },
    "Using": [ "Serilog.Sinks.MSSqlServer" ]
  },
  "WriteTo": [
    {
      "Name": "MSSqlServer",
      "Args": {
        "connectionString": "",
        "tableName": "Logs",
        "autoCreateSqlTable": true
      }
    }
  ]
}

Now in the dcoker-compose file we want pass the actual connection string:

  my-api:
    environment:
      - ASPNETCORE_ENVIRONMENT=Production
      - Serilog__MinimumLevel__Default=Warning
      - Serilog__WriteTo__0__Args__connectionString="Your connection string"

The value of each section can be accessed by __, for instance, Serilog__MinimumLevel__Default is equivalent to:

"Serilog": {
  "MinimumLevel": {
    "Default": "",

In a section to access an item inside the array, use the item index number. "WriteTo" section accepts an array of sinks configuration. If you are using two sinks use Serilog__WriteTo__0__ to access the first sink and Serilog__WriteTo__1__ to access the second sink.

Test

Let's do a simple test. Open a CMD or Powershell at the project directory:

• Type dotnet add package Serilog.Sinks.File to install File sink
• Open appsettings.josn file and change the logging configuration like this:

"Serilog": {
  "MinimumLevel": {
    "Default": "Debug",
    "Override": {
      "Microsoft": "Information",
      "System": "Information"
    }
  },
  "Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.File" ],
  "WriteTo": [
    { "Name": "Console" },
    {
      "Name": "File",
      "Args": {
        "path": "log.txt",
        "rollingInterval": "Day"
      }
    }
  ]
}

• Type dotnet build then dotnet run
• After running the application you should see a log file inside the project directory As you can see without touching codes we added another sink to save the log events.

If you're using SQL Server, PostgreSQL or MongoDB sinks, I have developed a small log viewer for small projects. It helps a lot, especially in the production environment and you don't need to query the database to view logs.

The source code for this walkthrough could be found on Github.