Why should you log and how to think about what to log.
An application's logs are the transcript of its life story, chronicling the salient moments and various turning points of its journey through time. Each application has its own unique story to tell, whether it is a mobile app, a distributed cloud service, or an ETL batch job. And, like all stories, some are better than others. One might be an interminable long-winded slog, and the next a joyless chronology of tedious minutiae. Some stories might even turn out to be both! But whereas readers may be forgiven for deciding they just don't care what ends up happening to Leopold Bloom, when it comes to your application's story, someone is probably going to have to buckle down and get through it at some point. That someone may be you!
So, as you consider how your application should write its life story, consider the needs of those who will be reading it down the road. Take into account their knowledge, skills, assumptions, and what they will want to get out of the experience. As they say, know your audience. We’ll go over a few of the different scenarios.
Logging for the Author (You)
The easiest requirements to gather are your own. Remember, however, the audience isn't exactly you—it's future you. So, how can we help future you to be more productive? Let's look at some reasons why future you might be reading your application's logs.
Investigating a Bug Report
If you're a developer, this is probably your primary use case. It's also the hardest to reason about. By its very nature, a bug is something you didn't expect to happen, so both the problem and its solution fall into the category of unknown unknowns. The best way to proactively assist yourself in debugging with logs is to ask yourself what information you would need in order to know which lines of code would be executed for a given input, and what their side effects would be. This is easier said than done, of course, but we’ll look at some specific examples in our post on debugging.
Improving Performance
Logs are not always the best tool for measuring performance, but they are a convenient first resort. We'll get further into this in our post on structured vs. unstructured logging, but one of the best things about an application log is the ease with which you can add a new log statement without worrying too much about breaking something. If you are using any kind of continuous deployment to deliver your application, you can answer simple performance questions with minimal effort and risk. One classic performance bottleneck is where your application needs to make a synchronous request to a database of some sort in order to complete a task. If the remote resource goes down or becomes unavailable, the problem is often discovered immediately. If the problem is more subtle, however, like in the case of a database query whose performance increases slowly over time as your data set grows, it can result in steadily decreasing application performance. One very simple approach to guard against this sort of performance problem getting out of hand is to log any database operation that takes more than a few seconds (or milliseconds, perhaps, depending on the composition and requirements of the system).
Estimating Impact
Imagine you have found, by inspection, a worrisome defect in a critical component. For the sake of this discussion, it doesn’t matter exactly what it is, but let’s say it would be “really bad” if a particular set of circumstances were to occur in production, and worse, the fix is not immediately clear. The first thing you’d want to know is whether, in fact, this issue is already occurring in production. You should add a test for these circumstances, log it, and redeploy ASAP. With the right search in your logging dashboard (and alerts), you should know very quickly whether this is a theoretical problem or an actual one. You will still probably want to address the issue, but with this information you can better strategize your way forward.
Preparing for a Change
Another thing we do all the time as developers is make changes to existing systems. Whether it’s a new feature, a bug fix, or an engineering refactor, sometimes a little extra information from your production environment can make a planned change go more smoothly. We make implementation decisions based on our own assumptions of how data and state flows through systems all the time, so why wouldn’t we want to check those assumptions before moving forward? If you’re ever at all uncertain as to whether real-world inputs to a function are a certain size, or contain a particular distribution of characters, for example, a new log statement can be a great way to quickly find out.
Logging for Other Developers (Not You)
The scenarios in which developers who are not you are sent scurrying to view your application’s logs are very much the same as your own (debugging, etc.). The main difference between them and you is that they are not you, and they don’t necessarily know everything you know. This may seem obvious to you now, but it will be even more infuriatingly obvious to the poor programmer who has to figure out what went wrong when they see a log message like this:
2021-04-01 01:23:45,678 - oops! something went wrong.
Don’t do this to your coworkers. Help them to understand how your code works as well as you do by giving them the context you already carry around in your head. For example:
2021-04-01 01:23:45,678 - Server.start(): failed to start because the server configuration file could not be found
This is only a very simple example, but note that the log message explains what encountered the error, where it happened, and a description of what the error was.
Non-Developers
Sometimes you will need to use your logs to answer questions for people from parts of the business outside of development. Very commonly this will include Dev Ops, who need to be able to run and scale your application, making decisions about production infrastructure without access to developers at all hours of the day. You probably also have Support staff, who need to be able to investigate customer issues and respond to end users without constantly escalating to engineering. You may even have a Legal department who needs to be able to document whether certain events have or have not occurred. A few well placed log messages can take care of these non-development needs as they arise.
Product Management / Sales
While logging is probably not the best tool for product and business analytics, sometimes it can do in a pinch. A few well placed log messages can be used to answer product and user behavior questions, and it is an option worth considering, especially if you do not or can not use an analytics package for some reason. As an example, you could keep track of REST API usage almost trivially by logging the request URI in combination with a user or session identifier of some sort. For another, you could log mobile application launches and whether or not the launch was initiated from a push notification and configure your logging dashboard to produce a simple user engagement report without very much effort at all.
Tools
Sometimes your logs aren’t consumed by a human at all, but by some kind of automated tool. In this case, your audience is both the user of the tool and the tool itself, which will often have its own very particular requirements. This is usually very specific to your own environment and best to check internally about which tools might be used.
Wrapping Up
The above are just a few types of users and use cases. There are likely many more varied and unique ones in your own organization.
If you are struggling to find new insight into how your application logs might be used by others, think of your logs as just another interface to your application, and the consumers of those logs as another user. With that mindset, you can address the question of what to log through the same product development processes you're already familiar with. Without prescribing any particular methodology, consider what it would look like to involve the consumers of your logs in that process, whether through formal requirements gathering, conducting interviews, modeling use cases, writing user stories, or some other ritual.
Hopefully this gives you a good overview of who will be reading your logs and why. In the next post, we’ll go into more detail as to exactly what you should log with each and every message.
Top comments (0)