Towards a fully programmable inbox

At Monolist, we're building the command center for engineers. We integrate with all the tools engineers use (code hosting, project management, alerting), and aggregate all their tasks in one place.

While we're constantly shipping new integrations, we understand that engineers may have specific workflows or internal tools that we can't natively integrate with. In this post, we'll go over using the new Monolist API to build a personalized workflow and get one step closer to automating your inbox.

The problem

Let's say we have an app with mobile and web clients. Sometimes, because of bugs, these clients end up in request loops that flood our app servers and cause downtime. Now, obviously we could solve this problem at various points in the stack -- we could introduce throttling at the client level, at the load balancer level, or at the application level.

For now, let's assume that we introduced some throttling at the application level using something like this library for Rails, or this one for Node.

Once we've shipped these changes we will want to monitor their behavior. How many users are we throttling? What is the false positive rate? How does the added limiting improve service availability?

Obviously, we could integrate our throttling directly into a more sophisticated monitoring system. We could (and should) build out dashboards that measure number of throttled users, and alerting for when excessive throttling takes place. However, this would require additional effort, and we may want to add this only after we've smoke tested the system. Is there a simpler solution?

Enter Monolist

Perhaps we simply want to be notified when a user is being throttled. We could send ourselves an email, or text message, but this notification is likely to be lost in the flood of emails/texts. On the flip side, we don't (yet) want to send an alert using something like PagerDuty, because we're not sure how frequently this will be happening, or what the correct severity should be.

Instead, let's use the Monolist API to add a task to our Monolist every time a user is throttled.

Note: The code samples in this blog post use Ruby and the HTTP library, but the Monolist API is accessible through any language.

def push_to_monolist(user)
  HTTP.post(
    "https://api.monolist.co/v1/action_items/new",
    json: {
      api_key: "API_KEY",
      action_item: {
        content: "Investigate throttled user #{user.id}",
        description: <<~DESCRIPTION
          Links
            * [Mixpanel](https://mixpanel.com/users?user_id=#{user.id})
            * [Logdna](https://logdna.com/logs?time=#{Time.now})
        DESCRIPTION
      }
    }
  )
end

Now we will be notified in Monolist along with our other tasks. We can prioritize and schedule this task the same as any other Monolist item. Because Monolist descriptions support Markdown, we have nicely formatted links that we can use to diagnose the issue.

One-click actions

At this point, you may be thinking, "couldn't I just have sent myself an email?"

The difference between Monolist and an email/text or any other type of notification, is that with the Monolist API, any task you create can be resolved directly in-line with fully customizable actions.

Let's say that when we investigate a throttled user, we want to notify the ops on-call rotation for any users that are not false positives. This way, they will be able to have context on any downtime implications.

With the Monolist API, we can add custom actions directly to our custom items. When a user takes a custom action, we can specify a destination url that Monolist will POST a payload to.

Let's use the PagerDuty API to directly create an incident:

def push_to_monolist(user)
  HTTP.post(
    "https://api.monolist.co/v1/action_items/new",
    json: {
      api_key: "API_KEY",
      action_item: {
        content: "Investigate throttled user #{user.id}",
        description: <<~DESCRIPTION
          Links
            * [Mixpanel](https://mixpanel.com/users?user_id=#{user.id})
            * [Logdna](https://logdna.com/logs?time=#{Time.now})
        DESCRIPTION
      },
      actions: [
        {
          name: "Escalate to Ops",
          url: "https://api.pagerduty.com/incidents",
          headers: [{ "Authorization" => "Token token=#{PAGERDUTY_TOKEN}" }],
          payload: {
            incident: {
              type: "incident",
              title: "User #{user.id} throttled",
              service: { id: PAGERDUTY_OPS_SERVICE_ID },
            }
          }
        }
      ]
    }
  )
end

When we create this new item, Monolist will encrypt and store the payload and headers objects. In the UI, the user will be able to take an action "Escalate to Ops"

When the user takes that action, Monolist will POST the payload to the PagerDuty API, automatically creating an incident for the Ops team.

Conclusion

In this post, we used the Monolist API to push a custom item with custom one-click actions into an user's Monolist.

Interested in building your own integrations? Check out the docs here.