Remote debugging of Java apps in Kubernetes

#kubernetes #java #debugging #docker

"It works on my machine ¯_(ツ)_/¯" - the older generation of developers often used this phrase in daily life. With Docker and Kubernetes there are platforms that solve such problems today.

But don’t be too happy about it: sometimes your application does not do what was implemented and tested locally. An experienced developer would turn on the debugger in such a case, but is that so easy on a Kubernetes cluster? Yes, and we'll show you exactly how in this blog post!

For the sake of simplicity, we will use the IntelliJ development environment throughout this post, but that should not be a big limitation - other development environments offer similar features.

Concept

The underlying idea is to enable debugging on a port in the Kubernetes pod (in which the Java app is running). This port must be exposed by the Pod.

Afterwards, a local port must be forwarded to the open debugging port of the pod using Port Forwarding. Last but not least, JVM Remote Debugging must be started in the development environment with the forwarded local port. The following diagram shows the structure of remote debugging in a Kubernetes cluster:

Concept — Figure 1: The concept of remote debugging in a Kubernetes cluster

Step 0: Setting up JVM remote debugging in IntelliJ

The first step is to set up remote debugging in IntelliJ. To do this, open please the configuration window for Run Configurations:

Editing Run Configurations — Figure 2: New "Remote JVM Debug" Run Configuration

Create a new Run Configuration by clicking on the plus symbol in the upper left corner and selecting Remote JVM Debug from the list.
Then you can give the configuration any name you like.
The standard configuration usually does not need to be adjusted. If the default port 5005 is used elsewhere in your application, you can change it in the configuration.
Now copy the command line arguments, they will be needed in the next step.
Finally, save the configuration, because you will need it later.

Step 1: Enable debugging in the remote JVM

We usually package our applications in Docker Images, which we build with such a Dockerfile:

FROM eclipse-temurin:17-alpine

ARG ARTIFACT
COPY $ARTIFACT app.jar

ENTRYPOINT ["sh", "-c", "java $JAVA_OPTS -jar /app.jar"]

Important is the variable $JAVA_OPTS in the ENTRYPOINT. We use this to insert the previously copied command line arguments. We do this by adapting the deployment resource in Kubernetes. With kubectl get deployments you can display a list of your deployments and with kubectl edit deployment <DEPLOYMENT_NAME> you can adjust the deployment as follows:

spec:
  containers:
    - ...
      env:
        - name: JAVA_OPTS
          value: <COPIED_JAVA_OPTS>
      ports:
        - name: remotedebugging
          containerPort: <PORT>
          protocol: TCP
      ...

Replace <COPIED_JAVA_OPTS> with the copied command line arguments from IntelliJ.
Replace <PORT> with the port you use for debugging, in our case 5005.

Once you have done this, you can close the editor and Kubernetes starts a new deployment for you. You can check this with the command kubectl get pods. A new pod should have been created. Important for the next step: Copy the name of the pod!

Step 2: Port Forwarding

After the Pod has been restarted, the debugging port of the Pod must be forwarded to your local machine. This can be done with the following command: kubectl port-forward <POD_NAME> <LOCAL_PORT>:<POD_PORT>

<POD_NAME> is the name of the pod you should have copied in the previous step.
<LOCAL_PORT> must be replaced with the port used locally for forwarding and to which the development environment connects for debugging.
<POD_PORT> is the port you opened in the deployment configuration in the previous step.

Step 3: Happy Debugging!

Finally, the debugging configuration created in Step 0 must be started and the following message should appear: Listening for transport dt_socket at address: 5005. 🎉

Now you can set breakpoints and find some bugs!