When you google log management tools, an interesting thing happens. At the time of this writing, you see no fewer than 4 paid ads, followed by a series of posts. These include, and this is not a joke, a post that lists the top 47. As a software developer and tools consumer, this drives me insane. It probably does the same for you.
An author named Barry Schwartz coined a term (along with an eponymous book) for this frustration. He called it "the paradox of choice," and it describes how, while we like to have some choice and autonomy, too much paralyzes us. To understand this in simple, terms, imagine selecting music for a dinner party. If offered two albums from which to choose, you'd make a pretty quick choice. If offered hundreds, you might thumb through them for a long time, trying to consider the likely tastes of all of your guests. And you might actually just give up eventually, and opt for only conversation with no background music at all.
The Paradox of Choice Among Log Management Tools
Back in the DevOps world, you face a similar plight when trying to pick among log management tools. You understand that you need a better way to aggregate and mine your logs than "by hand, using Sublime Text," so you start to do some research. And then, about two searches in, you find yourself staring at post entitled, "The Top 47 Log Management Tools." And, if you're anything like me, you rub your temples and say to yourself, "ugh, never mind, I'll figure this out tomorrow."
That, of course, lines up with Schwartz's findings about human behavior. Beyond having a few options, each additional option presented to a group of people causes fewer people to participate. The higher the number of log management tools in those posts, the fewer people will actually pick any of them at all.
Luckily, there's a path back to joy. And it's not even terribly complicated. You just need to dramatically narrow the field.
So today, I'm not going to add to the pile of "pros/cons/features" posts out there comparing dozens of tools. Instead, I'll speak to heuristics you can employ to help you choose among log management tools. I'm going to help you narrow the field from a paralyzing number of choices that you make you unhappy to a manageable number that empowers you.
Look to Those You Trust
Bar none, the most effective way to narrow a field involves relying on people and sources that you know and trust. I'm not talking about ratings sites ala Yelp, either. I'm talking specifically about colleagues and industry authorities that you follow and trust.
Ask them for their recommendations. What do they use and why? Do they like it? Would they recommend it? And, in terms of who you follow, do they have favorite tools? Does someone you admire work for one of the log management tools companies? Do you like their participation in the community?
Depending on the size of your network and reading sources, you'll get a list of varying sizes. Take this list, and set it aside for later cross-referencing.
"Wait," you're probably saying. Shouldn't this be the first way of filtering out the noise? You might think that, but the issue is that this list will be based entirely on the recommenders' needs and not on yours. Instead, set this list aside and go back to the wider field of potential options.
Narrow First with Pricing Clarity and Buckets
First things first. It might be a little gauche, but let's be frank. Cost matters, and it matters a lot. But I would advise you not to get overly concerned with the specifics of price. Instead, I'd slice things broadly into three buckets. (Actually four, but think of one as the null bucket -- I'll explain momentarily).
Reason about price by looking at tools as free, priced for small business, or priced for the enterprise. If you do not want to pay, you have the easiest way to narrow the field. Simply sort through the universe of options discounting any without a free or freemium option. If you reasonably think that you'll have budget for this, but not a lot, look for modestly priced tools (up to a few hundred dollars per month). If you work for a large enterprise, you know who you are. Assume that you'll want the feature-rich, higher-end options with lots of support. And, "market price" options where they just say to call about pricing fall into the enterprise bucket.
That leaves only the cryptic null case. What I'm talking about here is byzantine pricing schemes designed to confuse you. You know what I'm talking about. It happens when you stare at a pricing page for 10 minutes and, with all the rules, caveats, discount codes, and whatnot, you still can't figure out what it actually costs. Pricing should be honest and straightforward -- if you find yourself confused, cross it off your list and move on.
Disqualify Technical Mismatches
At this point, you've probably culled the field of log management tools down to roughly one-third of its original size on the basis of your appetite for spending. It's time now to slice it further by disqualifying obvious technological mismatches.
This can include the obvious, such as a tool that only installs its agent on Windows servers when you run Linux. But look too for features that you absolutely need. Is it only worth your budget if the tool offers a nice dashboard? Well, then make sure the tool has that dashboard.
I would caution against getting too restrictive about features, though. It's one thing to look a platform compatibility and a few essentials. It's another altogether to have a giant laundry list of "critical features" -- you can wind up eliminating all your options.
Optimize for Ease of Use
Hopefully, by this point you've narrowed the field considerably. That's important, because this last piece of research is a little more involved. You wouldn't want to do it for dozens of different prospective tools.
Set about now filtering tools based on their ease of use. You can figure this out by doing some research on their sites (or anywhere that you can find guides/demos of the products). Look for the install guide. Is it quick and easy, or is it involved, demanding tons of prerequisites and coordination? Next, look to see if they demo an install anywhere, like with a video. If that looks straightforward, you're in good shape.
Of course, you can also evaluate this by actually trying it yourself. That's a little more time consuming, but it speaks to the point of this line of research. Namely, once you've narrowed the field enough, you really just need to try using the tools to see if they work for you. Reading about APIs, libraries, platforms, and tools is one thing. Getting your hands dirty is another, and only that is going to really tell you whether it's a fit.
So if you've sliced your list down considerably and left only the easy to install options, you'll be in a position to try a few out. And, better yet, you're in a position to pivot from one to another if you find in the early going one isn't a fit. You can get going without worrying that you've over-committed.
Decide by Revisiting Your Whitelist
Now it's time to dust off that initial whitelist of recommendations. It's at this point that you've filtered your options down to the most likely candidates and are evaluating them in a meaningful way (via trial). To go back to the paradox of choice, you have now narrowed the field enough that the options empower you rather than paralyzing you.
Social proof, at this point, becomes powerful. Use your recommendations list as a potential deciding factor. Do you have three viable options, but only one of them comes recommended by a bunch of people you know and respect? That's a strong case for the recommended option, not only because of others' experience, but also because you'll have a support network for questions. Of course, your own experience with trying it is also powerful, so weigh those two factors together and decide.
There are so many tools in this space because the functionality is important and powerful. And having so many log management tools really is a wonderful position for consumers. But it's only wonderful if you know how to narrow the field to make your decision manageable.
Top comments (0)