DEV Community

Discussion on: A guide to Web scraping without getting blocked

Collapse
 
daolf profile image
Pierre

Thank you very much.

As long as we ensure that people don't use our service for DDOS purpose, we've been told we should be fine 🤞

Collapse
 
scosta profile image
Saul Costa

Hmm, interesting. Many sites list scraping, crawling, and / or non-human access as violations of their Terms of Service.

For example, see section 8.2(b) of LinkedIn's User Agreement (I list LinkedIn because I know they're a common target for scraping).

Thread Thread
 
daolf profile image
Pierre

Yes, you are right, Linkedin is well known for this.

Well, I am not a lawyer so I'd rather say nothing than saying no-sense.

We plan to do a blog post about this, well-sourced and more detailed than my answers :)

Thread Thread
 
scosta profile image
Saul Costa

Well, I am not a lawyer so I'd rather say nothing than saying no-sense.

Smart :)

We plan to do a blog post about this, well-sourced and more detailed than my answers

Cool, looking forward to reading that!

Collapse
 
andypiper profile image
Andy Piper

What about complying with terms of service for the websites and API platforms your service may scrape?