DEV Community

Cover image for Three Ways to Retrieve JSON from the Web using Node.js

Three Ways to Retrieve JSON from the Web using Node.js

Isa Levine on August 16, 2019

Cover image credit: Hunter x Hunter manga by Yoshihiro Togashi, meme-ified by yours truly. <3 In a recent technical challenge, I was asked to b...
Read full post
Collapse
 
three_ninjas profile image
Voted Jason Of The Year 2003-2007 •

Hi Isa, this is very helpful! I have a really dumb question...feel free to ignore me, I'm new to node and promises and kinda new to Javascript.

In the node-fetch example, what if what I wanted to do with the json was return it?

Like:
const getStuffFromWhatever = (item) => {
let url = whatever?thing=item

let settings = { method: "Get" };

fetch(url, settings)
.then(res => res.json())
.then((json) => {
    return json
});
Enter fullscreen mode Exit fullscreen mode

}

console.log(getStuffFromWhatever('puppy dogs')

I have tried a few things that seem reasonable to me, but they don't work. I could move the fetch into the function I actually need it in, but I will need it again elsewhere and that's not very DRY.

Collapse
 
three_ninjas profile image
Voted Jason Of The Year 2003-2007 •

Every time I put myself out there and ask a question I am embarrassed to ask, I always find the answer 5 minutes later, and this time is no different. The answer is async / await! Of course it is.

Thanks for letting me talk it out.

Collapse
 
isalevine profile image
Isa Levine •

Always happy to be your rubber ducky! :)

Collapse
 
gypsydave5 profile image
David Wickes •

Hey Isa - nice article! I think you picked the three ways I'd recommend for using an http client in Node.

A thought about the benchmarking of the the Node built in http lib: have you tried building a different data structure than a string - some sort of byte buffer - and then turning it into a string at the end. I'm not sure but I think it might be (I'm away from my computer so I can't try it myself).

Collapse
 
isalevine profile image
Isa Levine •

Hi David, thank you for the feedback! I'm looking into byte buffers (am I seeing this is originally a Java data structure?) for JavaScript, and I'm falling down a rabbit-hole about ArrayBuffers, Uint8Arrays, etc. I found my way over to the npm package bytebuffer (npmjs.com/package/bytebuffer), which looks like it provides a friendly API for using those structures.

I'm running up against a challenge of understanding how best to theoretically write to a byte buffer with incoming http request data--from this list of byte-buffer-writing options (github.com/protobufjs/bytebuffer.j...), do you have any advice on where I should be looking? I'm not sure which option will be more efficient than simply coercing the incoming data into a string.

Collapse
 
ihzaqstorm33 profile image
IHZAQSTORM33 • • Edited

Hi Isa,
Well can you explained little bit more further?
I mean look

const fetch = require('node-fetch');

let url = "https://www.reddit.com/r/popular.json";

let settings = { method: "Get" };

fetch(url, settings)
    .then(res => res.json())
    .then((json) => {
        // do something with JSON
});
Enter fullscreen mode Exit fullscreen mode

So you mean, do something with JSON is example:

console.log(JSON.kind)
Enter fullscreen mode Exit fullscreen mode

is that How to use it? thanks.

Collapse
 
geigle profile image
Geigle •

Thank you for this helpful guide. It got me far.
I was advised to use the new modern fetch() package.
developer.mozilla.org/en-US/docs/W...