APIOps: The art of mixing APIs

Good APIs encapsulate domain knowledge. You may use Twilio to embed multi-channel in-app communications, Stripe to process payments and AWS to host your applications. When you use an API, you let domain experts take care of the nitty-gritties while you get your work done. Much of our daily tasks today depend on calling multiple endpoints across different services in a single workflow.

In this article we will look at what involves APIOps and how you can do it with Postman. I will cover some of the lesser known features of Postman that will upgrade your APIOps game.

API-first Ops is for everyone

APIOps is a nice little term for performing tasks that involve APIs. It becomes significant in API-first organisations. DevOps teams need to spend more time and effort working with APIs to keep systems up and running. Testers need to work with API testing tools to ensure their APIs behave as expected. APIOps is not limited to engineering functions. It can span the entirety of an organisation — in sales, marketing, devrel, success, support. You name it.

Think of it in this way: If you use APIs to get things done, you are doing APIOps.

At Postman, we’re (web)hooked to Slack

The Postman team relies a lot on Slack for internal communications. Just like any organisation of our size and scale, we use an array of tools to get different tasks done. We use JIRA and Aha! for project management, Bitbucket to host code, AWS to host our platform, Sentry to track errors, Zendesk for support, Strikedeck for customer success, Salesforce to keep the sales pipelines alive and Looker for data science. The list goes on. Each function has its own needs and the right tools for the job.

Among all these tools, Slack is the one where everyone in the organisation is present. Slack has organically emerged as the de-facto medium for consuming reports at Postman, made possible by Slack API’s webhooks. It is almost a custom now for a new team member’s first Postman Collection to push the end result to Slack. If we were to run a scan to find the most common request across all of the collections that the Postman team creates, “Post to Slack” will definitely rank the first.

Protip: Postman’s growing collection of Templates has some interesting use cases for APIOps.

This practice has resulted in a rich set of collections that we use internally. We polish and publish some of those as templates in Postman for you to remix and reuse. We have collections which scan AWS IAM for security audit and send those results to Slack. We have a collection that posts every new comment from Postman’s GitHub issue queue to a Slack channel for everyone to see. We have another collection that alerts us of topics that have received no reply in our Discourse community forum. Among the fun ones, we have a collection that scans for nearest Sushi places using the Google Maps API and posts the findings as a Twitter DM.

The best part? All of these are programmable. You can hack on them. Remix them. Adapt them to your use cases.

Four aspects of APIOps

Whether your task at hand is to deploy an application, react to new entries, or to find the nearest Sushi bar, there are few aspects common to workflows involving HTTP APIs:

1. Interact with multiple APIs. Mix and match APIs across different services to fulfil a business requirement.
2. Chain API calls. You will want to trigger a group of HTTP requests one after the other. You will need to share data across these requests.
3. Trigger conditional workflows. Change execution order of requests based on responses. For example, you may want to call a webhook 10 times if you receive an array of 10 items from a previous request.
4. Make end result consumable. Send and compile reports of each execution to places where they can be consumed by others. This can be a Slack channel, an Airtable base, a remote database or an analytics system.

So, if you find yourself increasingly using APIs in your daily work, how will Postman help you? Postman today packs a lot more power than the simple HTTP client that it started off as. Let us dig in.

APIOps with Postman

Doing APIOps using Postman consists of a few common tasks. At a high level, you create a collection in Postman to send requests to the endpoints that you need; use environments to manage variable data; add scripts to the requests to perform the computations; use some other endpoint to send results to respective services.

Protip: If you are new to Postman, you would want to read through environments in Postman, variable scopes, scripting, pre-request scripts, tests, running collections and monitors.

A template for TL;DR

To make things simpler, I have created a collection called “APIOps 101” that covers all the features you need to know to do the 4 steps mentioned above, with a few examples thrown in. You can directly download the APIOPS 101 collection if you have the Postman app installed by clicking the button below.

The remaining sections of the article will elaborate on the tips and tricks mentioned in this template. You can download it and follow along, or just continue reading.

Collections — running and sharing them

Postman collections let you reuse workflows. You can add and organise requests in folders and subfolders within a collection. Creating a collection is the first stage of any APIOps you may do in Postman.

When you combine your requests in a collection, you build a workflow that can interact with different APIs. Each collection typically represents one workflow. Each folder in the collection represents a unit of the workflow. Think of it as building recipes for your favourite cuisines.

Collections are executable. When you run a collection, Postman triggers the requests in the collection one by one. This is the base of the automation you need for APIOps, allowing you to mix calls across APIs and chain those calls. You can run a collection from the Postman app or in CLI (using newman) or on a schedule using Monitors.

Once you have a few collections, you can collaboratively work on them by sharing them with your team members in team workspaces. That way, your collections (and, your API workflows) never go out of sync.

Protip: For more information on collections and how to run them, you can follow guided lessons in the app. Click the “Bootcamp” icon in the bottom bar of your Postman app to start a guided lesson.

Variables and their scopes

The next thing to know are variable scopes. Variables are the foundation of dynamic behaviour in Postman. You can use variables in requests, environment, collection and global scopes. In requests, variables use a double braces syntax. For example, {{url}} is a variable which you can substitute by declaring their value at any of the scopes. You can also edit them programmatically using scripts.

Variable scopes in Postman are resolved outside in. If you look at the diagram below, local variables have the highest precedence. Global variables have the least. Let’s look at them in brief:

• Global variables are valid for the entire Postman instance, multiple collections can share a single global variable.
• Collection variables apply to all the requests in that collection. The same variable set at a collection level will override the global variable, and will be available only for the scope of that collection.
• Environment variables are unique here. They override collection variables, but you can carry around an environment. Think of environments as a shareable container of variables that you can apply to multiple collections. You use pm.environment.get('key') and pm.environment.set('key', 'value') to get and set them through scripts.
• Local variables apply to a single request or to a single collection run. They override environment variables. You use pm.variables.get('key') to retrieve them and pm.variables.set('key', 'value') to set them through scripts.

Use local variables to transfer data between requests

Local variables come in very handy when you are building workflows using requests. You can transfer data between requests using either environment or local variables. The upside of using local variables in this context is that you don’t have to worry about your environments bloating up, since these variables will be reset after each run. It is also a more secure alternative to share sensitive data between requests.

The typical flow here is to process the response using scripts in the “Tests” tab of a request, set the required data as a local variable using pm.variables.set() and access it in next requests.

For example, your collection may have a request to fetch an authentication key and then use that same authentication key in subsequent requests. In this case, the first request can set the variable in the “Tests” section using something like pm.variables.set('authKey', authkey) (assuming authKey is the variable in your scripts). Subsequent requests can use the {{authKey}} variable in their headers or query params, wherever needed.

Protip: Variables can only store strings, so a common practice here is to JSON.stringify() the data before setting the variable. You can then use JSON.parse() after getting the variable later on.

Another common use case for local variables is to calculate values at runtime to add dynamic behaviour in a request. For example, you can use the following script to set the structure of a request to post data to Slack in the pre-request tab:

// Prepare the body for the Slack message
const slackBody = {
  "text": "\*Title here\*",
  "mrkdwn": true,
  "attachments": [
    {
      "title": "Field name",
      "text": "Some value"
    },
    {
      "title": "Another field name",
      "text": "Some more value"
    }
  ]
};

pm.variables.set("slackBody", JSON.stringify(slackBody));

You can then set the body of the same request to {{slackBody}}. Like this:

Use environment with Postman API to store states

You would often need to save states across collection runs. Collections runs are stateless. But, you can use environments to store data that you would need in subsequent runs. A common example of this is in processing lists of data, like parsing an RSS feed and sending new entries to Slack. You may need to remember which entries you have already processed so that you don’t trigger a notification for the existing one.

A quick and easy solution for this is to attach an environment to your collection runs and update the environment variables using pm.environment.set(). But, Monitors don't persist environment across monitor runs. This is where the Postman API comes handy.

You can use the “Update Environment” endpoint of the Postman API to update an environment with latest data. This can be a final request in your collection to save any relevant state by calling the Postman API. You can access the environment UID by switching to “Browse” view and opening the environment in the web dashboard.

This way, your collections remain portable across Monitors, in-app collection runner and newman.

Control execution order with postman.setNextRequest()

The postman.setNextRequest() method is one of the lesser used, but incredibly useful API that the Postman Sandbox exposes. You can use it in your script to branch out your execution order based on responses. This method takes one argument, the name of a request to execute after the current one.

Here are three very interesting things you can do with it.

1. Loop on the same request

You can pass the current request name as argument to the method to loop on the same request. Like, postman.setNextRequest("Process response array") where your current request name is "Process response array". You will need to ensure that you have some terminating condition to break out of the loop. For example:

// Assuming "results" in a stringified array
const results = JSON.parse(pm.environment.get("results"));

// Get the last item of the array
const currentResult = results.pop();

for (const item of currentResult) {
  // ... perform your action
}

// If there are more entries, loop on the same request.
// This is the terminating condition.
if (results.length) {
  // Stringify the remaining array items
  pm.environment.set("results", JSON.stringify(results));
  // Set the next request to the current one
  postman.setNextRequest("Post to Airtable");
}

2. Dynamically iterate on input data

Extending the previous point, you can loop on the same request to parse JSON or CSV inputs. Amber Race has written a nice article describing how you can loop on the same request to process CSV data inputs.

3. Stop executing further requests

You can stop executing next requests by passing null as argument to postman.setNextRequest(). Taking the previous example about parsing an RSS feed, you won't want to trigger a notification if there are no new items. In that case, the request that parses the RSS feed can set postman.setNextRequest(null), if there are no new entries. If the request to post to Slack is present after the current request, Postman will not stop execution and not trigger that request.

// Assuming `postsForSlack` is an array of entries,
// if there are any new posts to notify via Slack,
// call the next request, otherwise terminate collection run
if (!postsForSlack) {
  postman.setNextRequest(null);
  console.log("No new articles found.");
}

Write tests and set expectations

Finally, you would want to automate expectations. So, write as many tests as you need in the Tests tab of your requests and assert on the responses. That way, you will be able to ensure that all your responses and data structures are as you expect. You can read through the test scripts documentation, or try the quick snippets in the Postman app, or try the Bootcamp for automated testing.

With these tools in your toolkit, you will now be able to mix and mash APIs together as you wish. Go ahead and explore it! If you end up building something interesting, share those collections as templates for the rest of the community to remix. Happy APIOps-ing!

Cover Photo by Barn Images on Unsplash