Cleaning up after you are finished

Cleaning up is something you are used to tell your kids multiple times a day, however not many of us remember to do this with our resources.

As I write more kotlin and ktor apps, one thing that bit me recently was the fact that I needed to clean up some temporary resources after my app was terminated (scaled down) by kubernetes.

In my scenario, I was creating some temporary Google Cloud Pubsub subscriptions on startup, and I had to delete them during a scale down operation.

Reading the ktor's docs I found that it has built-in support for application events. It should be easy right?

Starting with Ktor

The simplest way to kickstart a ktor project is by visiting http://start.ktor.io, following the same principles of spring starter, it generates a project with all the dependencies and even some basic configuration files for you.

IMPORTANT

One change I make to all my ktor projects is enabling the fat jar feature using the shadow plugin. I prefer this method to distribute my apps. Just modify your build.gradle buildscript and plugins sections as bellow:

buildscript {
    repositories {
        jcenter()
    }

    dependencies {
        classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
        classpath 'com.github.jengelman.gradle.plugins:shadow:4.0.3'
    }
}

apply plugin: 'kotlin'
apply plugin: 'application'
apply plugin: "com.github.johnrengelman.shadow"

If you import your project on your IDE, your Main class should look something like this one (may differ depending on the features you chose at start).

fun main(args: Array<String>): Unit = io.ktor.server.netty.EngineMain.main(args)

@Suppress("unused") // Referenced in application.conf
@kotlin.jvm.JvmOverloads
fun Application.module(testing: Boolean = false) {
    install(ContentNegotiation) {
        jackson {
            enable(SerializationFeature.INDENT_OUTPUT)
        }
    }
    install(CallLogging) {
        level = Level.INFO
        filter { call -> call.request.path().startsWith("/") }
    }
    install(DefaultHeaders) {
        header("X-Engine", "Ktor") // will send this header with each response
    }
    routing {
        get("/") {
            call.respondText("HELLO WORLD!", contentType = ContentType.Text.Plain)
        }
        get("/json/jackson") {
            call.respond(mapOf("hello" to "world"))
        }
    }
}

You can now run ./gradlew build to package a fat jar on build/libs/<APPNAME>-<VERSION>-all.jar just run that jar file and you should see:

[main] INFO  Application - No ktor.deployment.watch patterns specified, automatic reload is not active
[main] INFO  Application - Responding at http://0.0.0.0:8080
[main] INFO  Application - Application started: io.ktor.application.Application@79517588

Enabling the lifecycle callbacks

As I said before we can subscribe for specific ApplicationEvents in our example let's watch ApplicationStarted and ApplicationStopped

If you change your main class to add two extra lines:

fun Application.module(testing: Boolean = false) {
    environment.monitor.subscribe(ApplicationStarted){
        println("My app is ready to roll")
    }
    environment.monitor.subscribe(ApplicationStopped){
        println("Time to clean up")
    }

And now you run it again:

[main] INFO  Application - No ktor.deployment.watch patterns specified, automatic reload is not active
[main] INFO  Application - Responding at http://0.0.0.0:8080
My app is ready to roll
[main] INFO  Application - Application started: io.ktor.application.Application@79517588

We can see that our callback has been called, but what happens when you terminate your process using a CTRL+C?

It turns out our callback does not get a chance to execute. So what went wrong?

Understanding how ktor bootstraps

Ktor has a couple of ways to bootstrap the server. The one used here uses EngineMain class as the entry point. This class will search for an application.conf file, in our example it looks like this:

ktor {
    deployment {
        port = 8080
        port = ${?PORT}
    }
    application {
        modules = [ io.igx.kotlin.ApplicationKt.module ]
    }
}

The module section defines how we configure the server, but the start/stop behavior of it resides inside EngineMain.

The Netty version (I have not tried this with other servers implementations) does not wait for the application to terminate in case of SIGINT and therefore it's impossible for our subscriber to be invoked.

How to fix

Another way to bootstrap a server is by using an embeddedServer function. And this is what we are about to do in order to fix this issue.

fun main() {
    val server = embeddedServer(Netty, port = 8080){
        module()
    }.start(false)
    Runtime.getRuntime().addShutdownHook(Thread {
        server.stop(1, 5, TimeUnit.SECONDS)
    })
    Thread.currentThread().join()
}

Note that we reused all the module configuration from the generated Application file, but now we bootstrap the server manually and by adding a shutdown hook that invokes the stop method on the server we can gracefully wait for our subscriber to run.

Before packaging and running this version, don't forget to change the mainClassName attribute on your build.gradle file.

Now running and killing it with a SIGINT:

[main] INFO  ktor.application - No ktor.deployment.watch patterns specified, automatic reload is not active
[main] INFO  ktor.application - Responding at http://0.0.0.0:8080
My app is ready to roll
[main] INFO  ktor.application - Application started: io.ktor.application.Application@327514f
START all SERVERS
[Thread-1] INFO  ktor.application - Application stopping: io.ktor.application.Application@327514f
Time to clean up
[Thread-1] INFO  ktor.application - Application stopped: io.ktor.application.Application@327514f

As you can see from the Time to clean up message in the logs, our method was successfully invoked during termination.

Final thoughts

This very simple changed meant a lot to my use case, cleaning up resources was very important specially on an elastic environment that instances can come and go many times a day.

I've reached out to the ktor team asking for this to be incorporated on the EngineMain method, hope they will add this feature on the future.

Happy coding