Control Rate Limit using Queues

Rate limiting is an important technique used to control the frequency of operations or requests within a specified period. It is commonly implemented to ensure system stability, fairness, and security

In this article, we are going to focus on how to use queues to control the rate limit usage of some external APIs.

External API rate limit

Imagine that you need to consume an external API that has a rate limit of 300 requests per minute, this means that the TPM (Transaction per minute) needs to be 300.

We are going to use Bulljs to define this rate limit as an example:

const queueWithRateLimit = new Queue('WITH_RATE_LIMIT', process.env.REDIS_HOST, {
    limiter: {
      max: 1,
      duration: 2000,
    },
  })

Bulljs enables us to configure rate limit per queue, if you need different rate limit requirements, you need to create more queues.

We made a calculateTPM function to make it easy to calculate the correct TPS and TPM

const calculateTPM = (limiter) => {
  const maxJobs = parseInt(limiter.max, 10);
  const maxDurationSeconds = parseInt(limiter.duration, 10) / 1000;

  const tpm = (maxJobs / maxDurationSeconds) * 60;

  console.log(`Your TP/M will be ${tpm}`);

  console.log(`Your TP/S will be ${tpm / 60}`);

  console.log(
    `This means that for every ${maxDurationSeconds} seconds, you will be able to process ${maxJobs} jobs.`,
  );
}

Using the config above it will give us the following output:

Your TP/M will be 300
Your TP/S will be 5
This means that for every 0.2 seconds, you will be able to process 1 job.

Briefly

Queues are a versatile tool to help you scale your distributed systems controlling the rate limit of external APIs, but also to constraint usage of resources.
What can be processed slowly, you can move to a queue with a lower TPM to avoid making your system heavy.

Woovi is an innovative startup revolutionizing the payment landscape. With Woovi, shoppers can enjoy the freedom to pay however they prefer. Our cutting-edge platform provides instant payment solutions, empowering merchants to accept orders and enhance their customer experience seamlessly.

If you're interested in joining our team, we're hiring! Check out our job openings at Woovi Careers.

Photo by Ludovic Charlet on Unsplash