DEV Community

loading...

Discussion on: Interacting with the Dev.to Article API

Collapse
turnerj profile image
James Turner • Edited

Cool, another post about the Dev.to API! 🙂

Correct me if I am wrong though but that will go over every single page till it runs out of data right? If so, you might want to be careful actually running that as there would be hundreds of pages and you're grabbing the contents for each article on every page. Maybe chuck a "max articles" count on there somewhere? Or maybe limit it to a specific tag?

Collapse
shindakun profile image
Steve Layton Author

The articles API endpoint looks to only return 30 articles per page and seems to cut off around ~460 pages or so. The idea is indeed to grab all public posts as I need a good amount of text for something else I'm toying around with so, in this case, I have a rough idea of how many articles we'll be getting. However, we're only getting 30 articles worth of content in one go which should not cause any issues on the server side or locally. You definitely wouldn't want to be running this repeatedly in a short span or anything though.

Collapse
turnerj profile image
James Turner

Yeah as it is still about 13,000 requests - if that ran often, it could easily hammer the site.

Thread Thread
shindakun profile image
Steve Layton Author

The articles endpoint is cached as far as I know so, as is the code shouldn't cause any significant load on Varnish. Heh. But yeah, imagine trying to simply download the articles one at a time, api/articles/1, that what like ~90000 hits. Yuck.

Forem Open with the Forem app