DEV Community

Discussion on: Secure your Data API from Web Scrapers

Collapse
 
mxldevs profile image
MxL Devs • Edited

Thanks for the feedback. Using browser based bots like puppeteer or selenium to simulate user behavior is quite effective, especially if the website is kind of annoying to scrape because it runs a lot of client side processing that you don't want to reverse engineer.

Though CORS really only protects you from browsers which also includes WebKit or webview based applications (eg: react) because the browser devs don't let you tamper with the origin header. Outside of regular browser contexts, CORS doesn't mean anything since you can supply your own origin header.

I've devised workflows for getting around CORS while still using development tools like react to build cross platform apps, and agree that CORS is generally quite effective for certain kinds of applications.