🦖 Gentle, promise-based HTTP client for Deno and Node.js (part 2)

GitHub deno.land/x npm

✅ DENO RU COMMUNITY

PART 1 (please review the first part before reading this article <3)

Hello! It's Vsevolod again 😎 In the first part, I outlined the problem: when working with various APIs, we often encounter limits and restrictions that the standard fetch function cannot handle. During my contemplation, I arrived at a solution (source code), which involved creating a queue through which all fetch requests would pass at a certain interval. Yes, this resolves the problem described above, but it is highly inefficient...(

❤️ And please, support the project by starring it on GitHub or subscribing to my Telegram
channel IT NIGILIZM!
This motivates me to continue developing this topic, writing new articles, and improving this tool!

🫠 Mistakes analysis:

So, what's the plan? Let's tackle this! The solution from the first part is inefficient because we're making N requests at equal time intervals, waiting for each request to complete. Let's assume we need to make 3 requests per second, no more, right? Take a look at the code from the previous article:

while (true) {
  await delay(interval);
  const req = queue.shift();
  if (!req) continue;

  try {
    const response = await fetch(req.url);
    req.resolve(response);
  } catch (error) {
    req.resolve(error);
  }
}

What could go wrong? Let's calc:

1st request ~300ms +
2nd request ~200ms +
3rd request ~250ms +
interval 1000ms = 1750ms!!!

So, it means that it will take us a long time to complete the first three requests, and the next three requests will only start after 1750ms. But we wanted to make 3 requests per second. What can be done about this? The most obvious solution is not to send requests sequentially, waiting for responses, but to do it "in parallel".

✨ New Solution!

Alright, let's start by adding a few new fields to HTTPLimiter (hereinafter referred to as "Limiter"):

1. requestsPerIteration - This field will serve as a counter, keeping track of the number of requests sent during one "iteration."
2. iterationStartTime - This is the start time of the batch request sending process in milliseconds.
3. loopIsWorking - A flag indicating whether the loop is active (I want to stop the request processing loop if the queue becomes empty). In IHTTPLimiterOptions, let's add rps: number (requests per second) for configuring the Limiter's loop.

Additionally, we will make some changes to the fetch function so that it triggers the request processing loop as needed:

  fetch(input: FetchInput, init?: RequestInit): Promise<Response> {
    const promise = new Promise((resolve, reject) => {
      this.#queue.push({
        input,
        init,
        resolve,
        reject,
      });
    });

    // Now we don’t have to run the loop “manually”
    // and leave it running in the background forever
    if (!this.#loopIsWorking) {
      this.#loopIsWorking = !this.#loopIsWorking
      this.#loop();
    }

    return promise;
  }

Great, now let's move on to the breakdown of the new loop. First and foremost, we need to remember the time when we started the loop and began sending requests. This is important because now we will make an honest effort to accurately count the number of requests per second:

async #loop() {
    this.#iterationStartTime = new Date().getTime();

The first thing to do when entering the loop is to check the number of requests we have already made. If it exceeds the allowed limit per second, we skip the iteration:

    while (true) {
      if (this.#requestsPerIteration >= this.#options.rps) {
        await delay(this.#options.interval);
        continue;
      }

Unlike the previous implementation, now fetch is executed asynchronously compared to other fetches in the queue. We don't wait for a response to the previous request to send the next one; instead, we simply increment the counter and decrement it when a response arrives:

      const entity = this.#queue.shift();

      if (entity) {
        this.#requestsPerIteration++;
        fetch(entity.input, entity.init)
          .then((response) => {
            entity.resolve(response);
          })
          .catch((error) => {
            entity.reject(error);
          })
          .finally(() => {
            this.#requestsPerIteration--;
          });
      }

Next, I establish the termination rule for the loop, namely:
if the queue is empty, we don't expect any more responses to the sent requests:

      if (!entity && this.#requestsPerIteration == 0) {
        this.#loopIsWorking = !this.#loopIsWorking;
        return;
      }

🤢 Here's a small workaround that prevents the request processing loop from blocking, which would otherwise hinder the handling of other promises:

      if (!entity && this.#iterationStartTime > 0) {
        await delay(0);
      }

And the last part of the loop, controlling the intervals. If we hit the limit on the number of requests, we calculate the time we need to delay before starting the next iteration. It's not a perfect solution, but it works:

      if (this.#requestsPerIteration == this.#options.rps) {
        const timeOffset = this.#iterationStartTime + 1000 -
          new Date().getTime();
        await delay(timeOffset);
        // !!! there are no guarantees that sent requests will be executed after this delay
        this.#iterationStartTime = new Date().getTime();
      }
    }
  }

Certainly, this is not a perfect solution: it can lead to many problems (for example, if the requests themselves have a very long delay) and errors. But now the loop operates much more efficiently than the approach in the first part.

🫶 In conclusion and for the future...

Thank you very much for reading the second part of the article! You can find the entire code on GitHub, and I've also created a port for nodejs, which is now available on NPM (all links below). I hope to create a good alternative to great modules like ky or axios, taking into account the limits and requirements of various APIs. I'll certainly appreciate any suggestions or pull requests.

GitHub deno.land/x npm