DEV Community

loading...
Cover image for 3 Simple ways to bypass API rate limits

3 Simple ways to bypass API rate limits

LordGhostX
Everyone's Friendly Neighbourhood Ghost | Python/Golang Developer | Developer Advocate
Originally published at lordghostx.hashnode.dev ・4 min read

What is an API?

Have you ever wondered how you send a message on WhatsApp and then someone somewhere in the world received that exact message and you know someone they communicated via a server of some sort? This was made possible via APIs. APIs serve as an intermediary between two software applications and allow them to communicate with each other to transfer information.

API stands for Application Programming Interface. Each time you use an app like Facebook, send an instant message or check the weather on your phone, you’re using an API of some sort.

A lot of times when we are working on APIs, we are limited to a small number of requests we can make but then asked to pay some amount of money to get unlimited access. Oftentimes, we try to optimize our project to make fewer API calls so you don't hit paywalls sooner and other times we pull out our cards to pay for the service but a lot of times we don't want to pay and start looking for cheaper (free) alternatives. But there's a trick to bypass API rate limits. This won't work for all cases but it'll get you far especially if the API provider is limiting via IP address and not API keys.

1 - Making use of Proxies

1*f9xMc4XzPrY4QZQN6OAA2w.png

A proxy server acts as a gateway between you and the internet. It’s an intermediary server separating end users from the websites they browse. While using a proxy server, internet traffic flows through the proxy server on its way to the address you requested. The request then comes back through that same proxy server (there are exceptions to this rule), and then the proxy server forwards the data received from the website to you.

What this all means is that, If my IP address is 123.456.789 and I make use of proxy server 234.567.890 to access https://www.example.com, My request will go from my IP address to the proxy server, then from my proxy server to the target domain, then from my target domain back to my proxy server and back to my PC. It's more or less a technique to hide your IP address.

This can be used to bypass API who track their rate limits using IP addresses, by sending your requests using different IP addresses, you bump up your rate limit to x * y where x is the rate limit and y is the number of proxies you have. E.g 300 requests per day having 10 proxies become 3000 requests per day. You can get proxies from https://free-proxy-list.net/ http://free-proxy.cz/en/ https://hidemy.name/en/proxy-list/ which can be used in your scripts e.g Python requests module has full support for proxies.

import requests

api_url = "https://example.com"
proxies = { 
    "http": http_proxy, 
    "https": https_proxy, 
    "ftp": ftp_proxy
}
r = requests.get(api_url, proxies=proxies)
Enter fullscreen mode Exit fullscreen mode

2 - Creating multiple API keys

2-1___Header_Key_Management.png

This is useful in cases where the API provider uses API keys issues by their service to track rate limits, all you have to do to bypass their limits is create multiple accounts and get multiple API keys. You can also change your IP address after every request in case they track IP addresses too.

3 - Making API requests client-side with JavaScript

javascript-illustration.png

This is the most interesting method and can prove quite useful a lot of times. I'm gonna give a personal experience using this method to bypass rate limits.

I was building an application that allowed users to get information from a search box provided and for analytics purposes, I wanted to store the IP address of every user along with their country, region, and a few other information that can be extracted from an IP address.

The API I was using to get IP address data was limited at 10K requests a month and I didn't want to take chances "if users will use this 10K times or not" I decided to:

  • make the API request with client-side JavaScript
  • converted the response to a string
  • added it to the form on the page with an input that had a type of hidden
  • then used JavaScript to submit the form
  • then used my backend to get this data and convert to JSON then extracted my desired data This would no longer be My limits are 10K per month but then My limits are x * 10000 limits where x = number of users
fetch("https://example.com")
.then(data=>{
    return data.text()
})
.then(res=>{
    hiddenFormField.value = res; 
    pageForm.submit()
})
Enter fullscreen mode Exit fullscreen mode

Summary

  • We learned what APIs are
  • We learned 3 ways to bypass API rate limits
    • Proxy servers
    • Multiple API keys
    • Client-side JavaScript
  • We learned what proxies are

I hope you’ve found this post to be helpful. If you know any other methods or want to suggest something which wasn't included, feel free to leave a comment! Happy Hacking!

Discussion (3)

Collapse
imthedeveloper profile image
ImTheDeveloper

Nice ideas πŸ‘ I've been using luminati.io with dedicated DC IPs for proxy. It cost me about $25 one off but never hit any reliability issues with proxies that I get with other solutions out there (namely free).

Collapse
lordghostx profile image
LordGhostX Author

Thank you for this recommendation, I'll check them out 😊

Collapse
nainglinaung profile image
Naing Lin Aung

The third option look like pretty promising and interesting to me.