DEV Community

Discussion on: Introduction to web scraping with Node.js

Collapse
 
aurelkurtula profile image
aurel kurtula

When @alexadusei asked I guessed it might not be doable to scrape dynamic content :). But now that I see that API (it says that you can "Crawl a SPA and generate pre-rendered content") I'll definitely try to figure this out

Collapse
 
alexadusei profile image
Alex Adusei

Yeah, very handy stuff. One technique people use (scraping AJAX is actually easier than regular scraping!) is using Google Developer Tools and going to the Network tab to see what external API calls the page is using. Then you can grab the information from there, plus more!

Thread Thread
 
aurelkurtula profile image
aurel kurtula

Aha, that's clever.