An Introduction to Microservices pt. 3

Table of Contents

1. What are we trying to solve?
2. The solution: Service Registry & Health Checks
3. Show Me the Code
   • Service Registry
   • Health Checks
4. Bibliography

Last time we talked about API Gateways. Today, we'll talk about two patterns that are very closely related to Gateways and each other: Service Registry and Health Checks.

What are we trying to solve?

When we created our API Gateway, we had to manually define the location (host+port) of the service, right? But imagine the following scenario: a given service needs to be scaled in the X-Axis. The new instances will have different locations and they must be made dynamically available to the clients.

Also, if one of our service instances becomes unavailable at any point, the Gateway must be aware that it can't route requests to that instance while it is unavailable.

The Solution: Service Registry & Health Checks

To solve our problems, we'll be using a combination of two patterns. First, let's discuss the Service Registry pattern.

The Service Registry is, as the name suggests, another service on our ecosystem, that is responsible to register all the services/instances of our application, and keep track of their location. By doing that, our Gateway can, any time that a request arrives, check with the Registry and route the request accordingly. This makes it invisible to the client which instance will receive its request and, as the instances change, the Gateway can always know valid locations! This is known as Server-side discovery.

Also, since services must cooperate with each other, they, too, can access the Service Registry to find the location of another service instance. This is known as Client-side discovery.

Benefits of Service Registry

• Makes it possible to the API Gateway to discover all services'/instances' locations dynamically

Drawbacks of Service Registry

• Unless the Service Registry is part of the infrastructure, it's yet another component that must be installed, configured e maintained;
• Since the Service Registry is a critical part of the application, it must have a high availability. If it becomes unavailable, the Gateway won't be able to router the requests.

But what if a registered instance becomes unavailable to receive requests? We shouldn't route anything to it, right? This is where our Health Checks pattern comes into play. As we register our services/instances, we'll also provide a way for the Service Registry to check if the registered instances are available to receive requests. If a instance is deemed 'unhealthy', our Gateway won't send any requests to it! That's awesome!

Benefits of Health Check

• It creates a way to check the health of the service/instance periodically.

Drawbacks of Health Check

• The check might not be comprehensive enough or the instance might fail between health checks, making so that requests are routed to an unavailable service/instance.

Show Me the Code

So, let's begin with the code needed to implement our Service Registry, service registration and health checks.

Service Registry

We'll begin by going the Consul.io website and downloading it. Consul will act as our Service Registry. Just for the purposes of this tutorial, we'll be running Consul in developer mode. After downloading Consul, you can add it to you system PATH, or run it from wherever directory you want it.

consul agent --dev

For our Service Registry, that's it. It's already running. You can check it by accessing https://localhost:8500/.

Now, let's move on to adjust our Gateway, so it'll check with our Registry for valid services/instances locations. We'll begin by editing our 'ocelot.json' file:

{
  "Routes": [
    "DownstreamPathTemplate": "/api/v1/{everything}",
    "DownstreamScheme": "https",
    "UpstreamPathTemplate": "/api/gateway/{everything}",
    "UpstreamHttpMethod": [ "GET", "POST", "PUT", "PATCH", "DELETE" ],
    "ServiceName": "YOUR-SERVICE-NAME"
  ],
  "GlobalConfiguration": {
    "ServiceDiscoveryProvider": {
      "Host": "localhost",
      "Port": 8500,
      "Type": "Consul"
    }
  }
}

As you can see, we've eliminated the "DownstreamHostAndPorts" property and added the "ServiceName" property to our route. Then, we add the "ServiceDiscoveryProvider" property and gave it the host, port and type of our Service Discovery. To complete this setup, we need to add another package to our project:

dotnet add package Ocelot.Provider.Consul

After it installs, just edit the "Startup.cs" file once again and add the following:

// Startup.cs
public void ConfigureServices(IServiceCollection services)
{
  // ...
  services.AddOcelot().AddConsul();
  // ...
}

That's it for our Gateway configuration. Now, we need to configure our services. They need to register with Consul, remember? That's what we'll do next.

For any service you'd like to register, you must add the Consul package to it.

dotnet add package Consul

After that, we need to do a couple of things. First, we need to add some needed configuration to our "appsettings.json" (we could put the information directly to the code, but this is better).

"ConsulConfig": {
  "Host": "http://localhost:8500",
  "ServiceName": "YOUR-SERVICE-NAME"
  "ServiceId": "AN-UNIQUE-ID-FOR-EACH-INSTANCE"
}

IMPORTANT: you must use the same "ServiceName" as you did on the routing in "ocelot.json" file. Also, the "ServiceId" is only needed if you plan to use multiple instances of the same service.

After that, we need to add Consul registration as a service on our Web API.

public static IServiceCollection AddConsulConfig(
    this IServiceCollection services,
    IConfiguration configuration
)
{
    services.AddSingleton<IConsulClient, ConsulClient>(
        p => new ConsulClient(
        consulConfig =>
        {
            var address = configuration.GetValue<string>(
                "ConsulConfig:Host"
            );
            consulConfig.Address = new Uri(address);
        }
        )
    );

    return services;
}

This will add the Consul client instance to our Web API. Now, we need to use the client instance to make our registration with Consul.

public static IApplicationBuilder UseConsul(
    this IApplicationBuilder app,
    IConfiguration configuration
)
{
    var consulClient = app.ApplicationServices
        .GetRequiredService<IConsulClient>();

    var logger = app.ApplicationServices
        .GetRequiredService<ILoggerFactory>()
        .CreateLogger("AppExtensions");

    var lifetime = app.ApplicationServices
        .GetRequiredService<IApplicationLifetime>();

    if (!(app.Properties["server.Features"] is FeatureCollection features))
    {
        return app;
    }

    var addresses = features.Get<IServerAddressesFeature>();
    var address = addresses.Addresses.First();

    Console.WriteLine($"address={address}");

    var serviceName = configuration.GetValue<string>(
        "ConsulConfig:ServiceName"
    );

    var serviceId = configuration.GetValue<string>(
        "ConsulConfig:ServiceId"
    );

    var uri = new Uri(address);

    var registration = new AgentServiceRegistration()
    {
        ID = serviceId,
        Name = serviceName,
        Address = $"{uri.Host}",
        Port = uri.Port
    };

    logger.LogInformation("Registering with Consul");

    consulClient.Agent
        .ServiceDeregister(registration.ID)
        .ConfigureAwait(true);

    consulClient.Agent
        .ServiceRegister(registration)
        .ConfigureAwait(true);

    lifetime.ApplicationStopping.Register(() =>
    {
        logger.LogInformation("Unregistering from Consul");
        consulClient.Agent.ServiceDeregister(registration.ID)
            .ConfigureAwait(true);
    });

    return app;
}

The code above removes any previous instances registered with the same ID, registers our service/instance and, finally, tells our Web API that, when it shuts down, it should deregister our service/instance.

And we're done! Now our services are registering with Consul and made available to our API Gateway. Let's begin to implement our health check!

Health Checks

To add our health check, we first need to install the "AspNetCore.HealthChecks.System" package. This will add some basic health check features. If you need to check the connection to your database, you'll need another package. In my repository, I used PostgreSQL, so, in my case, I needed to add the "AspNetCore.HealthChecks.NpgSql" package.

dotnet add package AspNetCore.HealthChecks.System

dotnet add package AspNetCore.HealthChecks.NpgSql

After the installation, we'll add the following code to our "ConfigureServices" method:

// Startup.cs
public void ConfigureServices(IServiceCollection services)
{
  // ...
  var builder = services.AddHealthChecks();

  builder.AddProcessAllocatedMemoryHealthCheck(
      500 * 1024 * 1024,
      "Process Memory",
      tags: new[] { "self" }
  );

  builder.AddPrivateMemoryHealthCheck(
      500 * 1024 * 1024,
      "Private memory",
      tags: new[] { "self" }
  );

  builder.AddNpgSql(
      Configuration.GetConnectionString("DefaultConnection"),
      tags: new[] { "service" }
  );
  // ...
}

We've just added some checks for the amount of memory our Web API is consuming (the limit is 500MB for both "Process Memory" and "Private Memory"), and a check for our database connection.

Finally, we need to create some endpoints to access this information. So, we'll add the following to our "Configure" method:

// Startup.cs
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
  app.UseHealthChecks(
    "/health",
    new HealthCheckOptions()
    {
      AllowCachingResponses = false,
      Predicate = r => r.Tags.Contains("self"),
      ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
    }
  );

  app.UseHealthChecks(
    "/ready",
    new HealthCheckOptions()
    {
      AllowCachingResponses = false,
      Predicate = r => r.Tags.Contains("service"),
      ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
    }
  );
  // ...
}

We've just added the endpoints "/health" and "/ready" to our Web API. Now, we need to tell our Service Registry that it should perform checks on the endpoints we've just created.

Let's go back to our "UseConsul" method and add the following:

var registration = new AgentServiceRegistration()
{
    ID = serviceId,
    Name = serviceName,
    Address = $"{uri.Host}",
    Port = uri.Port,
    Checks = [
      new AgentServiceCheck()
      {
        Name = configuration.GetValue<string>("ConsulConfig:ServiceName"),
        HTTP = $"https://{uri.Host}:{uri.Port}/health",
        Interval = TimeSpan.FromSeconds(30),
      },
      new AgentServiceCheck()
      {
        Name = configuration.GetValue<string>("ConsulConfig:ServiceName"),
        HTTP = $"https://{uri.Host}:{uri.Port}/ready",
        Interval = TimeSpan.FromSeconds(30),
      },
    ],
};

That's it! Now our services can register and tell the Service Registry about their health checks. Consul will perform the checks every 30 seconds and, if the check fails, the service will be marked as 'unhealthy' and won't receive any requests until it resolve its problems.

IMPORTANT: if your Consul instance can't perform the checks, try changing the 'https' to 'http' and the port to the 'http' port!

The next topic of the series is Load Balancer. Until next time!

Bibliography

1. Microservices Patterns: Examples with Java - Chris Richardson
2. Service registry pattern - https://microservices.io/patterns/service-registry.html
3. Client-side service discovery pattern - https://microservices.io/patterns/client-side-discovery.html
4. Server-side service discovery pattern - https://microservices.io/patterns/server-side-discovery.html
5. Consul by HashiCorp - https://www.consul.io/
6. Health Check - https://microservices.io/patterns/observability/health-check-api.html
7. Health checks in ASP.NET Core - https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/health-checks?view=aspnetcore-5.0