Recently I’ve been working on integrating with a subscription/payment gateway. (It hasn’t been straightforward, but that’s a whole other post…)
I wanted to be able to test my web-hook code without repeatedly triggering events from the gateway. I stored the incoming events in JSON format, which was fine - but then of course I needed to take the stored events and do something with them.
I thought it might be interesting to make a note of where I started from and how I got to the end. I’ve included the mistakes I made along the way, so if you read a bit and think “that won’t work!” - I probably found that out in the next paragraph. :-)
Starting out
Starting simple - read the file into an array of objects, and then loop through printing a couple of details from each event, so we know it loaded okay.
As this is test code I’m going to use the Sync version of readFile to keep the code simple - no callbacks, and we can feed the result of readFileSync straight into JSON.parse, like so:
const fs = require('fs');
function run() {
const json = JSON.parse(fs.readFileSync(__dirname + "/events.json"))
for (const event of json) {
console.log("event: ", event.id, event.event_type);
}
}
run()
Sure enough, we get what we expect.
$ node post-events.js
event: 1 Hello
event: 2 World
It works, but the loop is going to post the events really quickly. I’d prefer to space them out - it makes it easier to watch the receiving code that way, and I’m not trying to stress test it at this point.
Sending them gradually
setTimeout works nicely for queuing a function to be executed in the future. The simplest thing for the waiting time is to use the position in the array. The for...of construct doesn’t give us the index, so we’ll have to use a different method.
forEach can give us both the item and index, so let’s use that. It’s just the loop which changes - the file-reading and JSON-parsing stays the same, so I won’t repeat it.
json.forEach((event, index) => {
console.log(`Event ${event.id}: ${event.event_type}`);
console.log(`Will delay ${(index + 1) * 1000} ms`);
})
And yep, we’re good:
$ node post-events.js
Event 1: Hello
Would delay 1000 ms
Event 2: World
Would delay 2000 ms
Scheduling
Now we just need something to schedule. Let’s try the simplest thing first - for each event, queue a function taking the event as a parameter to print out the event id.
json.forEach((event, index) => {
const timeout = (index + 1) * 1000;
console.log(`Event ${event.id}: ${event.event_type}`);
console.log(`Will delay ${timeout} ms`);
setTimeout(event => console.log("Posting", event.id), timeout);
})
And:
$ node post-events.js
Event 1: Hello
Will delay 1000 ms
Event 2: World
Will delay 2000 ms
post-events.js:10
setTimeout(event => console.log("Posting", event.id), timeout);
^
TypeError: Cannot read property 'id' of undefined
at Timeout._onTimeout (post-events.js:10:52)
at listOnTimeout (node:internal/timers:557:17)
at processTimers (node:internal/timers:500:7)
After thinking about it that makes sense, and I really should have known better.
The event parameter is read when the function runs. Because of the timeouts the functions run after the loop has finished - at which point event is no longer defined, which is what we’re seeing.
Closure
What we can do is create what’s known as a closure. A closure is essentially a function together with the environment present when it was created. Luckily JavaScript makes that easy.
function makeFunc(event) {
console.log("Making func for", event);
return async function() {
console.log("Posting", event.event_type);
}
}
Yet another version of our loop:
json.forEach((event, index) => {
const timeout = (index + 1) * 1000;
console.log(`Setting timeout for Event ${event.id}; delay ${timeout} ms.`);
setTimeout(event => makeFunc(event), timeout);
})
$ node post-events.js
Setting timeout for Event 1; delay 1000 ms.
Setting timeout for Event 2; delay 2000 ms.
Making func for undefined
Making func for undefined
Well … something has gone wrong there. What’s happened is that because we wrote event => makeFunc(event), the call to makeFunc hasn’t happened straight away, but has been delayed - which gives us the same problem as before. Let’s make it an immediate call:
json.forEach((event, index) => {
const timeout = (index + 1) * 1000;
console.log(`Setting timeout for Event ${event.id}; delay ${timeout} ms.`);
setTimeout(makeFunc(event), timeout);
})
And see how that does:
$ node post-events.js
Setting timeout for Event 1; delay 1000 ms.
Making func for { id: 1, event_type: 'Hello' }
Setting timeout for Event 2; delay 2000 ms.
Making func for { id: 2, event_type: 'World' }
Posting Hello
Posting World
The POST request
That’s more like it. We’ll use axios for doing the POST to the HTTP endpoint.
const fs = require('fs');
const axios = require("axios");
const client = axios.create()
function makeFunc(event) {
return async function() {
console.log("Posting", event.event_type);
const res = await client.post("http://localhost:8000/", event);
if (res.isAxiosError) {
console.error("Error posting");
}
}
}
function run() {
const json = JSON.parse(fs.readFileSync(__dirname + "/events.json"))
json.forEach((event, index) => {
const timeout = (index + 1) * 1000;
console.log(`Setting timeout for Event ${event.id}; delay ${timeout} ms.`);
setTimeout(makeFunc(event), timeout);
})
}
run()
Looking at the output
You can use a service like requestbin as an easy way to check what POSTs look like. For this I’ve decided to use fiatjaf’s requestbin - it’s small and simple.
And here we are - correct data, and spaced a second apart as we expected.
$ ./requestbin -port 8000
Listening for requests at 0.0.0.0:8000
=== 18:00:00 ===
POST / HTTP/1.1
Host: localhost:8000
User-Agent: axios/0.21.1
Content-Length: 29
Accept: application/json, text/plain, */*
Connection: close
Content-Type: application/json;charset=utf-8
{"id":1,"event_type":"Hello"}
=== 18:00:01 ===
POST / HTTP/1.1
Host: localhost:8000
User-Agent: axios/0.21.1
Content-Length: 29
Accept: application/json, text/plain, */*
Connection: close
Content-Type: application/json;charset=utf-8
{"id":2,"event_type":"World"}
I hope that helps someone, even if it’s just that we ran into the same ‘oops’ and we made mistakes together. :-)
Latest comments (0)