Do you know of any alternatives for scraping sites that are dynamic/SPA's? I've heard that pupeteer github.com/GoogleChrome/puppeteer may be good for that?
When @alexadusei
asked I guessed it might not be doable to scrape dynamic content :). But now that I see that API (it says that you can "Crawl a SPA and generate pre-rendered content") I'll definitely try to figure this out
Yeah, very handy stuff. One technique people use (scraping AJAX is actually easier than regular scraping!) is using Google Developer Tools and going to the Network tab to see what external API calls the page is using. Then you can grab the information from there, plus more!
Do you know of any alternatives for scraping sites that are dynamic/SPA's? I've heard that pupeteer github.com/GoogleChrome/puppeteer may be good for that?
When @alexadusei asked I guessed it might not be doable to scrape dynamic content :). But now that I see that API (it says that you can "Crawl a SPA and generate pre-rendered content") I'll definitely try to figure this out
Yeah, very handy stuff. One technique people use (scraping AJAX is actually easier than regular scraping!) is using Google Developer Tools and going to the Network tab to see what external API calls the page is using. Then you can grab the information from there, plus more!
Aha, that's clever.