Beginners guide to logrotate.

Now that you have installed and setup Nginx in production for your web application. Your site is getting a good amount of traffic and all the logs are written in single file. This can be cumbersome to perform a certain task on file like searching and archiving the logs. Wouldn't be easy if can write daily logs on a separate file and compress the old logs and archive it separate place this will help us in optimizing the hard disk space.

Welcome to logrotate.

Logrotate is built-in ubuntu. With Logrotate we can do automatic rotation, compression, deletion and mailing the files. Each file can be rotated daily, weekly or monthly basis or when the file size becomes too large.

Main logrotate configuration is in /etc/logrotate.conf .If you look into that file which has configuration many default configuration. Before we dive into the configuration you should see below line in the file.

include /etc/logrotate.d/

Rather than writing all the process log into single file, let's create a separate configuration file so that it will be easy to maintain.First, we need to create an nginx conf file inside /etc/logrotate.d/

cd /etc/logrotate.d/
sudo touch nginx

Let's write the below basic configuration into that file and save it. I will run you through what each line does.

/etc/nginx/log/*.log {
  daily
  rotate 15
  missingok
  compress
  delaycompress
  notifempty
  dateext
  dateformat .%Y-%m-%d
  create 664 www-data www-data
  sharedscripts
  postrotate
    [ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid`
  endscript
}

/etc/nginx/log/*.log : Is the path of logs file you want to rotate.
daily : Rotate on a daily basis.
size : (100K/100M/100G) This will logrotate the log whenever the size of the file increases as configured.
rotate 15: Only last 15 log file are kept and others will be deleted. If rotate count is 0 old versions are deleted rather than rotated
missingok : If the log file is missing move to next file without throwing error.
nomissingok : If log file doesn't exist throw an error.
compress : It compresses the rotated file by default it uses gzip and file formats ending with .gz.
delaycompress: Delaycompress as the name suggest it delays compression of the log file so that previous processes after a restart or reload will be still written in the old log file so that we don't miss any logs. This will only work if you have compress in the configuration.
notifempty : Do not rotate the file if it's empty.
dateext : Archiving the old log file with date extension rather than by number. Format for a daily extension is YYYYMMDD. This is really handy when you need to search anything on a specific date.
create mode owner group: This creates a new log file with mode, owner, and the group immediately after rotation.
dateformat : To specify the date format extension to dateext.
sharedscripts : Sharedscripts are used with postrotate or prerotate, any scripts added to the postscript to run only once.
postrotate : Script inside the postrotate and endscript are executed after the logfile is rotated. This script does not kill the nginx process instead it sends a signal to reload the log files so that new requests are logged in the new file.
prerotate : Script inside this block will be executed before the logrotate.

This tutorial is just to get you started with logrotate. There are many configurations in logrotate like mailing the logs to the specific email address and many more. I encourage to check the inbuild documentation(man logratote)