loading...

Creating PULSE — A developer-friendly server monitoring tool (Part 4)

mattkingshott profile image Matt Kingshott 👨🏻‍💻 Originally published at Medium on ・5 min read

Facebook’s Prineville Data Server Center

Creating PULSE — A developer-friendly server monitoring tool (Part 4)

This is part of a weekly development blog series, where I will document the creation of an application from the initial idea through to its deployment on a scalable architecture. Even as an experienced developer, I find these stories to be interesting and I usually pick up a tip or two, so if you’d like to come along, and hopefully benefit in some way, let’s dig in!

NOTICE : Pulse has now launched and is available to use. You can create an account by visiting https://pulse.alphametric.co

Displaying log history

This week focused pretty heavily on creating charts for the purposes of displaying lots of data in a visual manner. Using charts allows a user to spot trends in a way they might otherwise not have, or spot them in a much faster manner since their eyes will naturally be drawn to colour over content.

However, before we examine the charting aspect of the app, I want to take a moment to discuss the database query being used to retrieve this data. When you have a query that could return potentially thousands of records, you need to implement some type of system to triage the data and extract a summary of content so that you don’t overwhelm the server.

Doing this in Laravel using Eloquent alone is not entirely possible out of the box, it instead requires a raw SQL sub query:

(
 SELECT id FROM 
 (
 SELECT [@var](http://twitter.com/var):=[@var](http://twitter.com/var)+1 AS rownum, id 
 FROM (SELECT id FROM logs ORDER BY id) AS sorted, 
 (SELECT [@var](http://twitter.com/var):=-1) AS var
 ) 
 AS ranked WHERE rownum % {{ x }} = 0
) AS subset

The query requires a value (supplied where x is), which is determined based upon the range of time between the start and end date for the filter. Here’s the code inside a PHP method (with the accompanying Eloquent query):

The log query method

So, how does this work? Well, essentially, it looks to see if the range of the filter exceeds thirty minutes. If not, it simply returns the logs for the range (as there will only be a maximum of 15 (one every 2 minutes).

If the range exceeds 30 minutes, the sub query is used. This performs the same query, but returns a maximum of 15 records which are evenly spread across the result set. In other words, if you had 1,500 records, you would get records 0, 100, 200, 300, all the way up to 1,500.

In essence, this provides you a summary of the data, which given we intend to display it in a chart, is more appropriate anyway.

As an aside, I’m not sure how the browser would handle trying to render a chart without such a triage system, but I think it would be painful, particularly if you were trying to do it on a mobile device.

Choosing a charting library

Now that I had the data available, I needed to select an appropriate JavaScript library to physically render the charts. There are many to choose from, but I ultimately selected Chartist because it offered a threshold plugin which I’d use for displaying when a server monitor status is positive or negative.

I also felt that the out-of-the-box UI was more attractive, and it offered good responsive design options, which would be critical for catering to mobile.

After some tinkering around with it, I came up with a design that worked nicely for showing the monitor’s status history:

Monitoring the MySQL service status (with red & green thresholds)

I was also able to include responsive options that allowed it to work nicely on mobile devices by reducing the number of labels and evenly spreading them. Here’s the same data presented on an iPhone:

A responsive design with Chartist

The nightmare of time zones

One of the realities of creating an application that has to deal with servers which are based all over the world, is that you need to handle the fact that they are operating in different time zones from YOUR server’s time zone.

So, how do you deal with this?

Well, fortunately, PHP’s Carbon library includes healthy time zone support, but you still have to do the conversion.

Let’s take a look at how the application works to see the workflow:

  1. The customer’s server sends a log to our application server.
  2. That log is stored with a time stamp using the UTC time zone.
  3. The customer then requests to view the logs between 3pm and 6pm.
  4. The application creates Carbon instances for those times using the time zone specified by the user for their server.
  5. The application then converts those Carbon instances to UTC for the purposes of performing the database query.
  6. The application then converts the dates returned by the query to the time zone of the customer’s server.
  7. We then display the data.

Here’s the PHP code used to parse the supplied date filters (including fall backs to the last twenty minutes if nothing is supplied):

The nightmare of time zone conversion

As you can see, it’s somewhat messy. There’s also a whole bunch of similar UI elements that require time zone conversion too. To that end, I’m considering creating some helper methods to make the code simpler in practice.

Wrapping Up

Well, that’s it for this week. Next up, we’ll be looking at the notification engine that Pulse will be using to inform users when there’s a problem.

We will also be examining the complexity that goes with this as we attempt to group notifications together, as well as implement intervals and daily limits to prevent the application from “spamming” users with too many advisories.

All that is coming in next week’s article. In the mean time, be sure to follow me here on Medium, and also on Twitter for more frequent updates.

NOTICE : Pulse has now launched and is available to use. You can create an account by visiting https://pulse.alphametric.co

Thanks, and happy coding!

Posted on Apr 8 '19 by:

mattkingshott profile

Matt Kingshott 👨🏻‍💻

@mattkingshott

Founder. Developer. Writer. Lunatic. Created Pulse, IodineJS, Axiom, and more. #PHP #Laravel #Vue #TailwindCSS

Discussion

markdown guide