DEV Community

Cover image for Know your Linux 06: Logs
Bassem
Bassem

Posted on • Updated on • Originally published at bassemmohamed.me

Know your Linux 06: Logs

Hello and welcome to the sixth and final entry in "Know Your Linux" series. This time let’s talk about Linux’s logging.

Before we dive right in 🏄‍♂️, make sure to check out the previous entry.

Why logging?

Well, Logging acts as your friend and advisor when the customer comes yelling WHY IS IT NOT WORKING?, It helps you understand the flow of the application and identify the causes of different problems. Without logging, It could be very hard to answer the big question What went wrong?

Logging in Linux

Linux - like other operating systems - provides logs on everything from kernel events to user actions. We can thank two services for that systemd-journald and rsyslog. All the collected logs by default are stored in the directory /var/log.

Wanna know more about what services and daemons are? checkout the previous entry.

Wanna know more about how Linux structures its directories?

Configuring logs

The rsyslog configuration file is /etc/rsyslog.conf, It has the following syntax :

facility.priority;facility.priority;... path
Enter fullscreen mode Exit fullscreen mode

For example :

*.info;mail.none;authpriv.warning; /var/log/logfile
Enter fullscreen mode Exit fullscreen mode

We have 3 terminologies at play here :

  • Facility: The type of application producing the log files, ex: mail, cron or authpriv
  • Priority: The severity of the log produced by a certain application
  • Path: Location of the log file where logs should be stored

We can override the /etc/rsyslog.conf in any file *.conf file under the /etc/rsyslog.d directory.

To prevent a certain facility from logging, the priority field can be set to none, which means that none of the logs from this facility will be added to the specified log file.

From the example above, We can see that the file /var/log/logfile will contain the following :

  1. *.info Logs with priority set to info from all facilities.
  2. mail.none None of the mail facility logs.
  3. authpriv.warning Only warnings from the authpriv facility.

Log Priorities

The table below shows all priorities and their meaning.

Code Priority Severity
0 emerg System is unstable
1 alert Action must be taken immediately
2 crit Critical condition
3 err non-critical error
4 warning Warning condition
5 notice Normal but significant event
6 info Informational event
7 debug Debug-level messages

Do logs stay forever?

Typically the log files are saved under /var/log which is a persistent directory but not forever. The service responsible for rotating the logs is called logrotate which is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large. Isn't it awesome? 🕶️

That’s all from my side ✋, That's also the last entry in "Know Your Linux" series, Make sure to check out previous entries. Also, Don't forget to tell me your thoughts in the comments bellow 🙄

As always,
Happy coding 🔥🔥
كود بسعادة”

Top comments (7)

Collapse
 
ferricoxide profile image
Thomas H Jones II

Probably want to qualify this statement:

Typically the log files are saved under /var/log which is a persistent directory but not forever.

Unless you've created /var/log as a pseudo-filesystem, everything under it is persistent. It's simply other processes that may be configured to help ensure you don't blow out whatever filesystem /var/log is sitting on.

While many of the log files that get placed in /var/log are accounted for in the default logrotate configurations, it's not safe to simply assume that things will be defaulted away (or, even if they are defaulted for rotation, your system's logging activity may happen too quickly for the default-settings to account for). A couple of things that can be gotchas:

  • The logs produced by auditd typically aren't rotated by the logrotate service
  • Depending on how you've set up your syslog – whether you've altered the default logging-paths or added non-default log-outputters – your log files very well may end up being wholly persistent …Even up to the point where your system becomes unusable or even crashses due to having run out of space.
  • Similarly, if you've configured any of your systemd units to handle their own logging activities (or installed applications that do similarly), you can likewise overflow your system
  • If you've installed software that don't log into /var/log (Splunk, McAfee and a few other "enterprise" agents are great for this)

In short, always understand what and where your software/services are logging, how quickly they're doing and update your log-rotation utilities configuration(s) to account for it. And, if you're paranoid (or operate under configuration-mandates), it can be helpful to make sure your logging directories are on separate filesystems from your / filesystem.

Collapse
 
bassemibrahim profile image
Bassem

Yeah! That's for sure. Thanks for clarifying!!

Collapse
 
cduv profile image
DUVERGIER Claude

What about "user actions"? I see no guide/example about this in the article.

I was looking for something like Windows Auditing of Windows Events such as login/logoff, file creation/deletion/modification, etc. but for a Linux OS.

Collapse
 
bootcode profile image
Robin Palotai

What tripped me, why call it maillog, if exactly the mail logs are missing :)

Collapse
 
bassemibrahim profile image
Bassem

Ohh, Sorry about that! 😂 Will fix it now!

Collapse
 
ggenya132 profile image
Eugene Vedensky

Came for the article, stayed for the beautiful timex banner.

Collapse
 
bassemibrahim profile image
Bassem

Sure, Whatever works! 😂 Happy that you liked it. To be honest tho. I don't really like it. But I feel like I gotta stick with the one banner that series started with.