DEV Community

Bidhan Khatri
Bidhan Khatri

Posted on • Originally published at bidhankhatri.com.np

Binding privileged port 514 to Logstash 7.10.0

#devops #beginners #webdev #linux

Sometimes there comes a scenario where you need to receive syslog traffic in the default port 514 in your logstash server. But as we know ports in range 1 to 1024 are privileged ports and only the root user can listen to them. By default, logstash daemon starts with its own user, so to resolve our problem we need to follow any below option. My production logstash server is on CentOS 7.

To bind logstash below 1024 ports we have 3 options.

  1. run logstash as root.
  2. use iptables to forward port 514 to an unprivileged port.
  3. use setcap to grant java permission to use privileged ports.

We could run the Logstash daemon through the root user but this process is not recommended and the process is also more complicated in the newer version of logstash so we gonna escape that option. You may prefer the 2nd option which is through iptables but their is limitation. If our logstash server is receiving millions of packets then every packet going through iptables rule will definitely increase our server CPU usages and we may face a performance bottleneck. So, therefore, we will choose the 3rd option in our case.

I will install logstash-7.10.0 through YUM repo which is bundled with JDK 11. That means I don't have to install java manually for my CentOS 7 anymore before logstash installation. Logstash offers architecture-specific downloads that include AdoptOpenJDK 11, the latest long term support (LTS) release of JDK.

Install logstash 7.10.0

Download and install the public signing key:

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
Enter fullscreen mode Exit fullscreen mode

Add the following in your /etc/yum.repos.d/ directory in a file with a .repo suffix, for example logstash.repo

vim /etc/yum.repos.d/logstash.repo

[logstash-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
Enter fullscreen mode Exit fullscreen mode

And your repository is ready for use. You can install it with:

yum install logstash
Enter fullscreen mode Exit fullscreen mode

Now check bundled java version.

/usr/share/logstash/jdk/bin/java -version
Enter fullscreen mode Exit fullscreen mode 
openjdk version "11.0.8" 2020-07-14
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.8+10)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.8+10, mixed mode)
Enter fullscreen mode Exit fullscreen mode

Add capabilities to the java binary

What is capabilities?
(According to it's man page)
For the purpose of performing permission checks, traditional UNIX implementations distinguish two categories of processes: privileged processes (whose effective user ID is 0, referred to as superuser or root), and unprivileged processes (whose effective UID is nonzero). Privileged processes bypass all kernel permission checks, while unprivileged processes are subject to full permission checking based on the process's credentials (usually: effective UID, effective GID, and supplementary group list).

Starting with kernel 2.2, Linux divides the privileges traditionally associated with superuser into distinct units, known as capabilities, which can be independently enabled and disabled. Capabilities are a per-thread attribute.

CAP_NET_BIND_SERVICE: Bind a socket to Internet domain privileged ports (port numbers less than 1024).

The capabilities are added per file. This is why we need to modify the java binary itself. The capability we need to add is CAP_NET_BIND_SERVICE, which is explicitly defined as the capacity for an executable to bind to a port less than 1024.

First, locate the java path in logstash. It's on /usr/share/logstash/jdk/bin/java

setcap CAP_NET_BIND_SERVICE=+eip /usr/share/logstash/jdk/bin/java
Enter fullscreen mode Exit fullscreen mode

Now check that the capability is added:

getcap /usr/share/logstash/jdk/bin/java

java = cap_net_bind_service+eip
Enter fullscreen mode Exit fullscreen mode

ldd will show shared library dependencies of an executable binary java

ldd /usr/share/logstash/jdk/bin/java
Enter fullscreen mode Exit fullscreen mode 
linux-vdso.so.1 =>  (0x00007ffd85fcf000)
libz.so.1 => /lib64/libz.so.1 (0x00007faa3d2c5000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007faa3d0a9000)
libjli.so => /usr/share/logstash/jdk/bin/../lib/jli/libjli.so (0x00007faa3ce98000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007faa3cc94000)
libc.so.6 => /lib64/libc.so.6 (0x00007faa3c8c6000)
/lib64/ld-linux-x86-64.so.2 (0x00007faa3d6de000)
Enter fullscreen mode Exit fullscreen mode 
systemctl start logstash
Enter fullscreen mode Exit fullscreen mode

Now you will see error message in /var/log/message saying libjli.so missing library. Java couldn't find that library.

logstash: /usr/share/logstash/jdk/bin/java: error while loading shared libraries: libjli.so: cannot open shared object file: No such file or directory.
Enter fullscreen mode Exit fullscreen mode

Above setcap command breaks how java looks for it's library to run. To fix this, we need to symlink the library it’s looking for into /usr/lib and then run ldconfig

ln -s /usr/share/logstash/jdk/lib/jli/libjli.so /usr/lib/
Enter fullscreen mode Exit fullscreen mode

Execute command ldconfig.

ldconfig
Enter fullscreen mode Exit fullscreen mode

Now create a test config file /etc/logstash/conf.d/test.conf listening on port 514.

input {
    syslog {
        port => "514"
    }
}

output {
    stdout { codec => rubydebug }
}
Enter fullscreen mode Exit fullscreen mode 
systemctl restart logstash
Enter fullscreen mode Exit fullscreen mode 
netstat -plnu
Enter fullscreen mode Exit fullscreen mode 
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
udp        0      0 127.0.0.1:323           0.0.0.0:*                           676/chronyd
udp        0      0 0.0.0.0:514             0.0.0.0:*                           26999/java
udp6       0      0 ::1:323                 :::*                                676/chronyd
Enter fullscreen mode Exit fullscreen mode

That's all. We have now successfully bind privileged port 514 to logstash.

In case if you need to remove capability then use setcap command again with the option -ep

setcap cap_net_bind_service=-ep /usr/share/logstash/jdk/bin/java
Enter fullscreen mode Exit fullscreen mode

Top comments (1)

Collapse
 
markbarry profile image
Mark Barry •

Nice post! An alternative to using a symlink and then ldconfig, is to create a config file and then run the ldconfig command.

 echo /usr/share/logstash/jdk/lib/jli/ > /etc/ld.so.conf.d/ld.logstash.conf
ldconfig
Enter fullscreen mode Exit fullscreen mode

I find this approach to be cleaner. The configuration can be validated with ldconfig -v.

# ldconfig -v
ldconfig: Can't stat /libx32: No such file or directory
ldconfig: Path `/usr/lib' given more than once
...
/usr/share/logstash/jdk/lib/jli: (from /etc/ld.so.conf.d/ld.logstash.conf:1)
        libjli.so -> libjli.so
...
Enter fullscreen mode Exit fullscreen mode

(the output is truncated)

Read next

ksolves profile image

Simplifying Real-Time Data Ingestion with Apache NiFi

Noel Campbell -

ravikishan profile image

Introducing NexaPHP: A Lightweight MVC PHP Framework

Ravi Kishan -

mikeyoung44 profile image

AI Boosts Math Construction Problems: PatternBoost Finds Elegant Solutions

Mike Young -

mikeyoung44 profile image

GenXD: The AI System Generating Realistic 3D and 4D Scenes Without Complex Modeling

Mike Young -