I wanted to call an endpoint updating an entity and I wanted to do it for 500 entities.
The API server had "rate limit" and I couldn't just use Promise.all since it will call the endpoint in parallel and the rate-limit would block my calls.
So I decided to write a script to bulk call the endpoint in series and have a 2 seconds wait before each call to make sure API server would not block them.
As I did that and it worked great, I thought it'd be advantageous to share!
Is there a better way for doing this?
How I'm doing this is starting from 0 and then adding up to that based on the array of data that I want to update. In the chain of promises that I have, I just added a delay before the API call, every time that I add 1 to the value that I'm passing to update the data till that number would be equal to my array length.
import fetch from "node-fetch";
import { data } from "./data.js";
const bearerToken = ""
const updateData = (id) => {
return fetch(
`https://www.url/${id}`,
{
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${bearerToken}`,
},
}
);
};
const delay = () => {
console.log(`Waiting: 2 seconds.`);
return new Promise((resolve) => {
setTimeout(() => {
resolve("2");
}, 2000);
});
};
const startTime = Date.now();
const doNextPromise = async (id) => {
delay()
.then((x) => {
console.log(
`Waited: ${x} seconds now calling the endpoint for updating data ${data[id]}`
);
return updateData(data[id])
.then((res) => {
if (res.status !== 200) {
throw `Error updating data ${data[id]}: ${res.status}`;
}
return res.json();
})
.then((res) =>
console.log(`Response: ${JSON.stringify(res)} for data ${data[id]}`)
)
.catch((e) => console.log(`Error: ${e}`));
})
.then((res) => {
id++;
if (id < data.length) doNextPromise(id);
else console.log(`Total: ${(Date.now() - startTime) / 1000} seconds.`);
});
};
doNextPromise(0);
Latest comments (2)
This doesn't bypass the rate limit at all - quite the opposite in fact. It makes your calls obey the rate limit
Good point! I guess that was my mindset when I was doing that since it was blocking me. But you’re right it is in fact obeying the rate limit to make those calls