DEV Community

Dev-suite
Dev-suite

Posted on

All You Need To Know About ProxyCurl

Image description

Proxycurl is Distributed Crawling Software that helps developers save time and money by helping them save resources.

Make use of Proxycurl API to:

It is possible for Proxycurl to scrape 1000 web pages a day without exceeding a website's rate limit, or encountering Recaptcha, or bot detection issues. The Proxycurl API section in the dashboard has a section where each user is assigned a randomly generated token that will be used for verification. To be processed, each eligible request needs at least one credit. If and only if the request is correctly parsed a credit is used.

Up to 300p queries per minute are permitted through our API. There is a 5-minutes window for the rate cap. Thus, you can send up to 1,500 requests in a burst every five minutes. To keep our services available to all customers during times of high load, our system may tighten rate limits for every account. When You are rate limited, we return an error status code of 492. If our ability is limited you can potentially receive a status code error status of 429. In addition to using exponential back-off, you should handle 429 errors.

To learn more about Proxycurl API kindly check the link below:

https://nubela.co/proxycurl

Top comments (0)