DEV Community

Aleksander Parchomenko
Aleksander Parchomenko

Posted on • Edited on

Ocelot & files download

Intro

One of the common patterns when you design your microservices based apps is an gateway pattern in opposite on client-to-microservice communication. You can write your own gateway applications or use third party solutions for it. One of the widely used solution in .NET world that Microsoft recommends is Ocelot.

It is convenient and simple solution for creation API gateways.

Problem

During of one the recent projects I've faced and interesting problem for one of the microservices that was responsible for files downloading. For files that size was greater than about 250 MB Ocelot gateway returned HTTP Content-Length Header equals to zero whereas microservice actually returned desired file content. The result of it was an empty downloaded file on client side.
I've found out that it is existing and reported old (dated on Jun 8, 2020) Ocelot issue: 1262.
There is also another very similar issue but solution in comments does not work in my case.

For our application solution of the problem was to override the default HTTP client used in Ocelot. I've done it only for one endpoint that is responsible for file downloading just to keep all the implemented Ocelot logic with caching, logging, etc. for rest of configured API.
To do it I've needed to perform 3 steps:

  • To find default implementation of Ocelot IHttpRequester. It is a HttpClientHttpRequester
  • To implement custom IHttpRequester implementation based on HttpClientHttpRequester for contents endpoint (that is responsible for files downloads):
public class HttpRequester : IHttpRequester
{
    private readonly IHttpClientCache _cacheHandlers;
    private readonly IOcelotLogger _logger;
    private readonly IDelegatingHandlerHandlerFactory _factory;
    private readonly IExceptionToErrorMapper _mapper;
    private readonly IHttpClientFactory _httpClientFactory;

    public HttpRequester(IOcelotLoggerFactory loggerFactory,
        IHttpClientCache cacheHandlers,
        IDelegatingHandlerHandlerFactory factory,
        IExceptionToErrorMapper mapper, IHttpClientFactory httpClientFactory)
    {
        _logger = loggerFactory.CreateLogger<HttpClientHttpRequester>();
        _cacheHandlers = cacheHandlers;
        _factory = factory;
        _mapper = mapper;
        _httpClientFactory = httpClientFactory;
    }

    public async Task<Response<HttpResponseMessage>> GetResponse(HttpContext httpContext)
    {
        if (httpContext.Request.Path.ToString().Contains("contents"))
        {
            var client = _httpClientFactory.CreateClient("FilesDownloadClient");
            var request = httpContext.Items.DownstreamRequest();
            var response = await client.SendAsync(request.ToHttpRequestMessage(), HttpCompletionOption.ResponseHeadersRead);
            return new OkResponse<HttpResponseMessage>(response);
        }

        var builder = new HttpClientBuilder(_factory, _cacheHandlers, _logger);
        var downstreamRoute = httpContext.Items.DownstreamRoute();
        var downstreamRequest = httpContext.Items.DownstreamRequest();
        var httpClient = builder.Create(downstreamRoute);

        try
        {
            var response = await httpClient.SendAsync(downstreamRequest.ToHttpRequestMessage(), httpContext.RequestAborted);
            return new OkResponse<HttpResponseMessage>(response);
        }
        catch (Exception exception)
        {
            var error = _mapper.Map(exception);
            return new ErrorResponse<HttpResponseMessage>(error);
        }
        finally
        {
            builder.Save();
        }
    }
}
Enter fullscreen mode Exit fullscreen mode
  • Register of this custom IHttpRequester implementation and HttpClient factory in gateway DI container before registering Ocelot:
builder.Services.AddHttpClient("FilesDownloadClient", client =>
{
    client.Timeout = TimeSpan.FromMinutes(10);
});
builder.Services.TryAddSingleton<IHttpRequester, HttpRequester>();
builder.Services.AddOcelot(configuration);
Enter fullscreen mode Exit fullscreen mode

Now client can download any files you need through yours Ocelot gateway.

Important thing in IHttpRequester implementation is to have set HttpCompletionOption.ResponseHeadersRead (that means the operation should complete as soon as a response is available without buffering and headers are read. The content is not read yet.) during requesting underlying API:

var response = await client.SendAsync(request.ToHttpRequestMessage(), HttpCompletionOption.ResponseHeadersRead);
Enter fullscreen mode Exit fullscreen mode

You also have to remember that default timeout for HttpClient is 100 seconds and for large files gateway can throw an exception. To change timeout you can use:

builder.Services.AddHttpClient("FilesDownloadClient", client =>
{
    client.Timeout = TimeSpan.FromMinutes(10);
});
Enter fullscreen mode Exit fullscreen mode

Top comments (1)

Collapse
 
dexter_c828209f7f4219fb32 profile image
Dexter

Hi! and how would the implementation be to upload files up to 250mb?