Creating PULSE — A developer-friendly server monitoring tool (Part 7)

mattkingshott profile image Matt Kingshott 👨🏻‍💻 Originally published at Medium on ・4 min read

Facebook’s Prineville Data Server Center

Creating PULSE — A developer-friendly server monitoring tool (Part 7)

This is part of a weekly development blog series, where I will document the creation of an application from the initial idea through to its deployment on a scalable architecture. Even as an experienced developer, I find these stories to be interesting and I usually pick up a tip or two, so if you’d like to come along, and hopefully benefit in some way, let’s dig in!

NOTICE : Pulse has now launched and is available to use. You can create an account by visiting https://pulse.alphametric.co

Methods for retrieving data

Before we get into the technical implementation of how Pulse will gather the data it needs to determine whether a server has a problem, we should review the possible methods that Pulse could use:

  1. We could use SSH to tunnel into a server and then execute a series of commands and copy the responses into the database.
  2. We could require the installation of a monitoring agent (a fancy phrase for an app) to collect all of the data and then send it to Pulse.
  3. We could use a shell script that a user configures to run at a set interval. The script gathers the data and then sends it to Pulse.

Now, let’s take a quick look at each of these methods, and see what led me to take the approach that Pulse will actually be using.

Using an SSH tunnel

The pros of this approach are simple. If Pulse can connect to your server, then we know it isn’t silent. If we can connect, we can run commands as if we were sitting in front of a server terminal. It’s by far the most powerful option, and that’s ironically the reason it was never considered.

In order to SSH into a server, Pulse would need credentials to gain access to your server and (depending on the permissions) could do anything it wanted at that point. While Pulse obviously has no malicious intent, a user has no real way of knowing what Pulse COULD be doing and therefore it’s too much of an ask. Plus, the code to achieve it would be quite gnarly.

Using a monitoring agent

This is the most common approach used by data monitoring software, but it has its drawbacks. It requires you to add repositories to server, install the agent itself and finally configure it to run (and that assumes your server has the necessary dependencies installed for the agent to run).

In addition, the above steps require you to have root access on the server. If you manage the server yourself, then you’ll probably have this, but many developers won’t, and their server administrator may not be prepared to give them the credentials or permit the installation of software. In this situation, there’s no way around the problem. You are simply out of luck.

Using a shell script

This is, in my opinion, the ideal solution. It doesn’t require SSH access, it doesn’t require root permissions and it doesn’t require you to install any software or modify your server in any way (besides setting a cron entry).

Further more, you never need to concern yourself with what the script is doing, as you can simply open it in any text editor and see the commands that it will be executing. Perfect for server administrators that need to know exactly what code is running on their infrastructure.

Further benefits of a shell script

Since we’re not distributing a compiled app, we can dynamically generate a script that corresponds to the monitors the user has configured. In other words, it gathers data for only what you ask it to, nothing more, nothing less.

The second major benefit of this approach is flexibility. Suppose you had a server with a particularly unusual Linux distribution running on it and that the correct command to retrieve data for a monitor was different. If Pulse simply used a compiled agent, you’d be out of luck, but with a shell script, you can simply modify the command to ensure it returns the right data.

While doing this won’t be the norm for 90–95% of users, it’s great to know that you can do it if you needed to.

The script implementation

Pulse contains a table of monitors along with the shell commands used to retrieve the data for them. When generating a script, all it needs to do is look up the commands for each monitor configured for a server.

It then dumps them into the script line by line, merges the data into a JSON string and finally includes a cURL snippet to send the data off to Pulse.

A sample data collection script generated for a server

Scheduling the script to run

Once the user has copied the shell script over to a server directory of their choosing, all they have to do, is tell Linux to run the script at a regular interval so that Pulse can keep track of how the server is doing.

We can do this by registering the script with crontab using one command, which will fire every two minutes:

\*/2 \* \* \* \* /path/to/script.sh

Wrapping Up

Well, that’s it for this week. Next up, we’ll be taking a look at a feature I’m calling “Clusters” that will help you to organise your infrastructure.

All that is coming in next week’s article. In the mean time, be sure to follow me here on Medium, and also on Twitter for more frequent updates.

NOTICE : Pulse has now launched and is available to use. You can create an account by visiting https://pulse.alphametric.co

Thanks, and happy coding!

Posted on Feb 26 '19 by:

mattkingshott profile

Matt Kingshott 👨🏻‍💻


Founder. Developer. Writer. Lunatic. Created Pulse, IodineJS, Axiom, and more. #PHP #Laravel #Vue #TailwindCSS


markdown guide