DEV Community

Cover image for Project Idea 02 – journalctl Parser
Itachi Uchiha
Itachi Uchiha

Posted on • Updated on

Project Idea 02 – journalctl Parser

#ideas #sideprojects #journalctl #linux

This post published on my blog before

Hi everyone. Today I'm going to tell you about my project idea. Before this post, I published another one.

What's this journalctl?

Let's dig into journalctl man page using the below command;

man journalctl

We will see an output like that

journalctl may be used to query the contents of the systemd(1) 
journal as written by systemd-journald.service(8)
Enter fullscreen mode Exit fullscreen mode

So, it's a command to get systemd logs and it uses systemd-journald.service

What is systemd-journal service

systemd-journald is a system service that collects and stores logging data. It creates and maintains structured, indexed journals based on logging information that is received from a variety of sources. These are can be message types;

  • Kernel Messages
  • Simple Log Messages
  • Audit records

There are a lot of messages you can find.

Let's See Some journalctl Commands

If you're using journalctl without any parameters it will show full output;

journalctl
Enter fullscreen mode Exit fullscreen mode

The output

-- Logs begin at Sat 2020-01-18 21:00:40 +03, end at Sat 2020-05-09 10:47:50 +03
Jan 18 21:00:40 opcode kernel: microcode: microcode updated early to revision 0x
Jan 18 21:00:40 opcode kernel: Linux version 5.3.0-26-generic (buildd@lgw01-amd6
Jan 18 21:00:40 opcode kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-5.3.0-26-g
Jan 18 21:00:40 opcode kernel: KERNEL supported cpus:
Jan 18 21:00:40 opcode kernel:   Intel GenuineIntel
Enter fullscreen mode Exit fullscreen mode

You can get json output in pretty format

journalctl -o json-pretty
Enter fullscreen mode Exit fullscreen mode

The output

{
        "__CURSOR" : "s=a62023d453d2404c949ce66c81b4b97a;i=1;b=5bf547fda1f147129
        "__REALTIME_TIMESTAMP" : "1579370440473152",
        "__MONOTONIC_TIMESTAMP" : "5696941",
        "_BOOT_ID" : "5bf547fda1f147129ac28544e4d1b35f",
        "_SOURCE_MONOTONIC_TIMESTAMP" : "0",
        "_TRANSPORT" : "kernel",
        "PRIORITY" : "6",
        "SYSLOG_FACILITY" : "0",
        "SYSLOG_IDENTIFIER" : "kernel",
        "MESSAGE" : "microcode: microcode updated early to revision 0x27, date =
        "_MACHINE_ID" : "35bb650aeefb48379f3b1920848e2a5a",
        "_HOSTNAME" : "opcode"
}
// more pages here
Enter fullscreen mode Exit fullscreen mode

You can also get specific outputs for instance chrome's logs;

journalctl _COMM=chrome
Enter fullscreen mode Exit fullscreen mode

The output

-- Logs begin at Sat 2020-01-18 21:00:40 +03, end at Sat 2020-05-09 10:47:50 +03
Jan 20 20:49:09 opcode chrome[8566]: Failed to load module "canberra-gtk-module"
Jan 20 20:49:09 opcode chrome[8566]: Failed to load module "canberra-gtk-module"
Jan 20 20:49:10 opcode audit[8804]: AVC apparmor="DENIED" operation="sendmsg" pr
Jan 20 20:50:24 opcode chromium_chromium.desktop[8566]: [9131:1:0120/205024.3276
Jan 20 20:50:24 opcode chromium_chromium.desktop[8566]: [9131:1:0120/205024.3
Enter fullscreen mode Exit fullscreen mode

As you see, these are the oldest messages. What about the current boot's log or specific boot's messages?

To get a list of boots, use this command;

journalctl --list-boots
Enter fullscreen mode Exit fullscreen mode

The output

-92 5bf547fda1f147129ac28544e4d1b35f Sat 2020-01-18 21:00:40 +03—Sat 2020-01-18 
-91 f6a4dc011a8847bb94572a02de1c8401 Sat 2020-01-18 21:25:32 +03—Sun 2020-01-19
// more than this
Enter fullscreen mode Exit fullscreen mode

To see boot 91's message, use this command;

journalctl -b 91
Enter fullscreen mode Exit fullscreen mode

There are many commands you should know.

What Will We Do?

As we see journalctl useful command to understand system or application logs. But it's also hard to understand. You have to use a terminal, you have to know all commands. (In this idea you have to) but the end-user may don't want to know all commands.

We can write a parser in our best programming language. It can be a web project or another terminal project or GUI application.

Users can filter logs between two dates

To do this idea, use this command;

journalctl -S "2020-01-01 00:00:00" -U "2020-01-02 00:00:00"
Enter fullscreen mode Exit fullscreen mode
  • -S: since
  • -U: until

And search about this command

Users can filter logs by specific services

For example, you want to see logs for apache2 use this command;

journalctl -u apache2.service
Enter fullscreen mode Exit fullscreen mode
  • -u: unit

Users can filter logs by specific binary

For example, you want to see logs for chrome use this command;

journalctl _COMM=chrome
Enter fullscreen mode Exit fullscreen mode
  • _COMM: match for the script name is added to the query

Users can see all boots

I'm an end-user who wants to see all boots. But it's really hard to see for me. Use this command;

journalctl --list-boots
Enter fullscreen mode Exit fullscreen mode

Users can see logs from different boots

For instance, we want to see the logs for boot 35, we should use this command;

journalctl -b 35
Enter fullscreen mode Exit fullscreen mode

Users can filter logs by priority

To filter logs by priority use this command;

journalctl -p 0
Enter fullscreen mode Exit fullscreen mode

You can specify the number or level key.

journalctl -p crit
Enter fullscreen mode Exit fullscreen mode
  • -p: priority

These are log levels;

  • 0: emerg
  • 1: alert
  • 2: crit
  • 3: err
  • 4: warning
  • 5: notice
  • 6: info"
  • 7: debug"

Technologies

You can use various technologies to achieve this idea. For example, golang really good programming language. I believe you can do that in Python easily. I'll choose NodeJS to do that.

EOL

Actually these are my thoughts. You can extend them. Your project will have better features than my project's features.

Sorry for the grammar mistakes.

Thanks for reading ^_^ and if there is something wrong, tell me.

Resources

These resources helped a lot while thinking of this idea. I learned many new things. Remember that you can learn new things while thinking about something.

Top comments (0)

Read next

yunwei37 profile image

eBPF Developer Tutorial: XDP Load Balancer

云微 -

yunwei37 profile image

eBPF Practical Tutorial: Using eBPF to Trace Go Routine States

云微 -

yunwei37 profile image

eBPF Tutorial by Example: Capturing TCP Information with XDP

云微 -

vijay431 profile image

Streamlining AWS Secret Management: A Bash Script for Efficient onboarding of secrets into pipeline

Vijay Gangatharan -