loading...
Cover image for Reverse engineer an API using a web proxy

Reverse engineer an API using a web proxy

loopdelicious profile image Joyce Lin Originally published at Medium ・6 min read

When you think about reverse engineering an API, perhaps corporate espionage or something legally or ethically questionable comes to mind. Hackers gonna hack. However, there’s also legitimate reasons for reverse engineering an API.

Why reverse engineer an API

APIs aren’t always documented. When you’re debugging an API, you can diagnose and resolve issues more quickly when you can visualize all the data that is sent and received. Doing this allows you to gain a deeper understanding of an API.

Reverse engineering, also called back engineering, is the process by which a man-made object is deconstructed to reveal its designs, architecture, or to extract knowledge from the object.

Knowing how an API actually behaves enables you to identify flaws and security vulnerabilities like accidental data leakage. This also performance tests your API so you can isolate bottlenecks that could benefit from additional caching and compression.

Of course, there’s also selfish reasons for reverse engineering an API.

Selfish reasons for reverse engineering an API

When you’re reverse engineering a private API that isn’t yours, make sure to check the terms of service. Some developers wear their cease and desist orders like a badge of honor, but you might get sued or banned from their services.

What is an HTTP/S proxy?

A web proxy server is like a middleman that sits between your client application and the server. The web proxy is a substitute server that can intercept HTTP traffic between a web browser and the website’s real server.

It’s all there! Black and white, clear as crystal!
— Willy Wonka, The Chocolate Factory

Reasons to use a web proxy

  • Monitor and eavesdrop on HTTP network traffic by recording and displaying all traffic between your client and server
  • Hide your public IP address while browsing websites and to access blocked content
  • Filter or redirect requests to limit access to content or optimize systems performance

Postman is a proxy that captures the HTTP/S request

Free web proxy tools

  • Postman is a free tool with a bunch of different proxies, including a built-in proxy to capture HTTP requests in the native apps for Mac, Windows, or Linux
  • Mitproxy is an open-source proxy with a command line interface, web interface, and Python API
  • Fiddler is a free web debugging proxy with support for a wide variety of browsers, systems, and platforms
  • Burp has a free community edition of a web proxy server that lets you view and modify requests and responses

The remainder of this article will focus on how you can use Postman to intercept the traffic between your client and server.


A Postman recipe for reverse engineering an API

It’s time to get our hands dirty, sniffing and inspecting to our heart’s content. First, we’ll import a single request into the Postman app. Then we’ll use Postman as a proxy to capture a stream of HTTP/S requests from a variety of clients like a desktop web browser, a mobile device, and an Electron app.

To follow along in Postman, click the orange +New button in the top left of the Postman app. Under the Templates tab, search for Reverse engineering an API, and import the sample collection into your instance of the Postman app.

Import this collection and follow along with these examples

Read through the descriptions in the Postman app for details, or check out the web documentation for step-by-step instructions and screenshots.

Import a single request

If you want to inspect a request in Postman, here’s a super simple way to import it as cURL from your browser. In this example, we will use Chrome DevTools to inspect and select a request. You can follow similar steps when using other web browsers.

Copy the cURL request from Chrome DevTools

Paste the cURL request as raw text in Postman

Inspect a stream of requests

If you want to inspect a stream of requests from your client, you can use the Postman built-in proxy to capture these requests. Postman has a bunch of different proxies. In this scenario, we’ll rely on the Postman built-in proxy in the native apps for Mac, Windows, or Linux. You can capture requests sent from your client, like a desktop web browser, mobile device, or an Electron app.

You can pipe this stream of requests to your Postman History and save them to a Postman Collection. Then you can revisit these requests for inspection at a later date, or share them with a teammate.

Currently, the Postman built-in proxy in the native apps only captures HTTP request traffic. Fortunately, most websites have HTTP Strict Transport Security (HSTS) enabled for an additional layer of security. Unfortunately, this means the Postman built-in proxy cannot capture requests sent over HTTPS if the website has HSTS enabled.

Note: As of the publication of this article, the Postman built-in proxy captures HTTP traffic, but not traffic from HTTPS websites with HSTS enabled. Interceptor integration and HTTPS proxy is slated for development in the Postman roadmap for developers.


A final thought about reverse engineering an API

This recipe is just the tip of the iceberg of how you can sniff and inspect HTTP traffic to start understanding what’s going on under the hood. For tougher nuts to crack, you may have to dig into SSL certificate pinning, spoof particular client attributes, or learn how to sign and authenticate more complex requests.

Although these tools and methods are powerful and can be used for selfish reasons, they can and should be used for good.

Be excellent to each other.
Bill S. Preston, Esq.

Tools like Postman can enhance the visibility of client requests, making them easier to replicate and tweak, so you can diagnose and resolve issues faster. Ultimately, this will help you gain a deeper understanding of a public or private API, especially for APIs that aren’t well-documented.

This article was originally posted on the Postman Engineering blog

Posted on by:

loopdelicious profile

Joyce Lin

@loopdelicious

Puns and danks memes are kind of my wheelhouse

Discussion

markdown guide
 

Sometimes docs are not good enough. If you are testing getting at the real payload is valuable, and if you are poxying through wiremock then faking the service for testing.