Logging in your API
This article will highlight some methods and best practices you can adopt in your service for proper logging and error tracing.
Table of content
- What is logging
- Choosing your log framework
- What and when to log
- Try and catch?
- Data Safety
- Cloud Logging
- Glossary
- References
What is logging
As copied from Wikipedia Logging is the act of keeping a log of events that occur in a computer system, such as problems, errors or just information on current operations. These events may occur in the operating system or in other software.
Choosing your log framework
Most API frameworks like spring, .NET, laravel comes equiped with
robust logging packages that you can use in your project. The list
below highlights some logging framework you can use in your services.
- Java -> Logback, Log4j2, JDK (Java Util Logging), Slf4j, e.t.c.
- C# -> Built in Logger, Log4Net, Serilog, NLog, e.t.c.
- PHP -> Built in Logger, Monolog, Analog, KLogger, Log4PHP
- C/C++ -> liblogax, Boost.Log, log4c, e.t.c.
- NodeJS -> Pino, Winston, Bunyan, Npmlog, e.t.c.
What and when to log
When it come to loggin no information is too much to be logged and
there is no specific place to place a log just ensure that the logs
are properly classified.
There are 6 basic log classification
- TRACE
- DEBUG
- INFO
- WARN
- ERROR
- FATAL
When logging basic information that does not change on each call it can be classified as INFO log, if a logic or operation is likely going to cause an error in the service or a non fatal error occur then it can be classified as WARN log. For operations that cause an error that
breaks or stop the flow of an operation then it can be classified as ERROR log.
In the case of FATAL log the service is expected to end or shutdown,
a situation when a FATAL log is required is when the memory of the
system is completely used or corrupt a shutdown is expected cause it
will be almost impossible for the service to continuing function
properly after that.
Try and catch?
Exception plays an important part in logging as the error in the
exception catch block is logged to give the user an idea of what error
occur and possibly why it occur.
In strongly type language like Java, C# the try and catch logic
is robust enough to allow handling specific exceptions but in weakly
typed language like Javascript, Python, developers can become lazy in
using the language provided type check to determine what error is thrown
which in turn encorages generic and un-helpful logging. Then there is
languages like PHP that allows both strong and weak typing in special
case like try...catch block, for such languages always go for the strong
typing feature for safety and better error tracing.
Exception Handling
If you have a function or method that perform multiple operations, the
try and catch block should not be at the highlevel of function rather it should be at the last logic level such that when the exception is logged
it can easily be traced to the logic point.
An example of this is a method that accept a POST body, converts the body json string to object then store the object in the database.
BAD
public ResponseEntity<?> store(@RequestBody String body) {
try {
User user = mapper.readValue(body, User.class); // can throw JsonException
// some long operation ...
repository.save(user); // can throw ConstraintException
// some other long operation ...
return new ResponseEntity<>(HttpStatus.OK);
} catch (Exception ex) {
return new ResponseEntity<>(HttpStatus.INTERNAL_SERVER_ERROR);
}
}
GOOD
public ResponseEntity<?> store(@RequestBody String body) {
User user;
try {
User user = mapper.readValue(body, User.class); // can throw JsonException
// some long operation ...
repository.save(user); // can throw ConstraintException
// some other long operation ...
return new ResponseEntity<>(HttpStatus.OK);
} catch (JsonException ex) {
log.error("Error occur while processing body payload", ex);
return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
} catch (ConstraintException ex) {
log.error("Error occur while persisting to database", ex);
return new ResponseEntity<>(HttpStatus.INTERNAL_SERVER_ERROR);
}
return new ResponseEntity<>(HttpStatus.OK);
}
EFFICIENT
public ResponseEntity<?> store(@RequestBody String body) {
User user;
try {
user = mapper.readValue(body, User.class);
} catch (JsonException ex) {
log.error("Error occur while processing body payload", ex);
return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
}
// some long operation ...
try {
repository.save(user);
} catch (ConstraintException ex) {
log.error("Error occur while persisting to database", ex);
return new ResponseEntity<>(HttpStatus.INTERNAL_SERVER_ERROR);
}
// some other long operation ...
return new ResponseEntity<>(HttpStatus.OK);
}
Now let analyse the trinity above, take note of the long operation
comment.
- Bad: This prevent logging which error actually occur though it will catch any error even even those that we did not imagine occuring at the long operation blocks. It makes the service not debuggable as it will almost be difficult to trace the error down do where it actually occurs. And also a custom message cannot be returned to the caller.
- Good: In the good block we take account of the two types of error that can occur but we have our other code logic (long operation) into tha try..catch block which is not neccessary.
- Efficient: In this code block we only put a try catch around each logic which will spare the other logics (long operation) from being caught in the try..catch block.
Logging Exception For Some Weakly Typed Language
In languages that does not provide specific exception catching, it is
best practice to check what exception occur so as to produce proper log.
Take for example the code block below in Javascript and PHP, handles
general error that can occur in a try catch block.
- Javascript
try {
// operation 1 can throw JsonParseException
// operation 2 can throw DatabaseException
} catch (err) {
console.error("Error occur", err);
return response.status(500);
}
- PHP
try {
# operation 1 can throw JsonParseException
# operation 2 can throw DatabaseException
} catch ($e) {
echo "Error occur: " . $e->getMessage();
return response->status(500);
}
The issue with the exception catch above is that, two type of exception is likely to occur but both of them are handled in the catch block and
if a response is expected from the caller a generic response is giving
which is not likely to be helpful to the caller. Therefore it important
to check the type/instance of that error to properly log the error and
send meaning ful response to caller.
- Javascript
try {
// operation 1 can throw JsonParseException
// operation 2 can throw DatabaseException
} catch (err) {
if (err instanceof JsonParseException) {
console.error("Error while parsing Json body", err);
return response.status(400);
} else if (err instanceof DatabaseException) {
console.error("Unable to persist to database", err);
return response.status(500);
}
console.error("Unknown error occur", err);
return response.status(500);
}
- PHP
try {
# operation 1 can throw JsonParseException
# operation 2 can throw DatabaseException
} catch ($e) {
if ($e instanceof JsonParseException) {
console.error("Error while parsing Json body", $e);
return response->status(400);
} else if ($e instanceof DatabaseException) {
console.error("Unable to persist to database", $e);
return response->status(500);
}
console.error("Unknown error occur", $e);
return response->status(500);
}
In strongly typed language like Java you can handle the two exceptions
in seperate catch blocks. And there is no need for the general
exception case since it unlikely to occur.
try {
// operation 1 can throw JsonParseException
// operation 2 can throw DatabaseException
} catch (JsonParseException ex) {
log.error(ex.getMessage(), ex);
return response.status(400);
} catch (DatabaseException ex) {
log.error(ex.getMessage(), ex);
return response.status(500);
}
Almost say no to eception message
When it comes to logging the exception's .getMessage(), or .message
method or fields are almost useless as a properper logging needs to
throughly output where error occur, why it occur and possible what
caused the error to happen. The message value of exceptions in this
case does not carry all the informations rather it only returns
what error occurs.
Hence it proper to log the entire stack trace of the exception.
Logging your exception
If the framework or logging framework used in your project provides
methods to log stack trace then you should use that instead of printing
the stack trace directly to your console. This allow the log
framework to categorize the exception and possibly sanitize the stack
trace before printing to the console.
For example In java spring rather than using ex.printStacktrace(),
invoke the error method in any of the provided log implementation e.g.
Logger log; // = any log implementation
try {
//...
} catch (DatabaseException ex) {
// bad
ex.printStackTrace();
// good
log.error(ex.getMessage(), ex);
}
Data Safety
When it comes to logging it is important to be careful of the data
being sent to the log service or printed in the console. Information
like secrets, passwords should NEVER be logged rather these type
of information can be printed out as *** or in an encrypted value
that only the developer can decrypt the actual value.
A good example is a simple User class in spring if there is a need to
print out the class field ensure the secrets/password fields are
hidden or encrypted in the log.
class User {
String username;
String password;
@Override
public String toString() {
return "[User { username=" + username +", password=******}]";
}
}
Cloud Logging
Most API frameworks come equipped with robust logging framework which
you can take advantage of when implementing logging in your service.
An example is the spring framework which have the cloud logging
feature to enable tracing log and fails between multiple services.
Configuring Spring Sleuth
For Spring Framework the cloud logging is provided through the
spring cloud package. Detailed walkthrough and version of spring
cloud that matches your spring or spring boot version can be seen
here
Glossary
Strongly Typed: A strongly typed programming language is one in which each type of data, such as integers, characters, hexadecimals and packed decimals, is predefined as part of the programming language.
Stack trace: This is the collection of stack records which stores the
procedure or method call information up until the stack trace is
invoked. Information stored in stack trace are the file name, line
number, column number e.t.c.Weakly Typed:
Top comments (0)