I’ve been in the business of data extraction and web scraping for more than 10 years now. As the owner of Price2Spy - a price monitoring tool, capable of scraping not only prices but much other information from websites I got complaints and some website owners even threatened me with legal consequences if I don’t stop gathering data from their websites.
This leads me to my question - how damaging to your business do you find competitors who are scraping data from your website?
I recently started building a new product - BotMeNot - a bot protection score service https://botmenot.com/. Do you think this would be helpful for your anti-bot efforts?
Top comments (2)
I have to say, I really respect your hustle. Like an arms dealer selling weapons to both sides.
I've never had a problem with people scraping my sites, after all the sites are there to provide information and scraping is just another way people gather that information. I've always told clients, if your business relies on putting information on the web while not wanting competitors to have that information, then you don't really understand the internet and probably shouldn't be running a web based business in the first place.
As for blocking bots, well, at best it's rude and I tend to block and ignore sites that block my ability to use the information they provide.
Thanks for your feedback!