DEV Community

Martin Hauner
Martin Hauner

Posted on • Originally published at hauner.github.io

fixing a (transitive) dependency mess with gradle

#gradle #java

the transitive dependency mess

While extracting openapi-processor-core from openapi-processor-spring I noticed duplicate dependencies with different and sometimes unexpected versions in the external libraries section o the project view (in IDEA).

In this case jackson packages.

Running gradle dependencies to see where they are coming from prints the following output (look for jackson):

gradle dependencies (simplified)

+--- com.fasterxml.jackson.module:jackson-module-kotlin:2.11.0
|    +--- com.fasterxml.jackson.core:jackson-databind:2.11.0 <1>
|    |    +--- com.fasterxml.jackson.core:jackson-annotations:2.11.0
|    |    \--- com.fasterxml.jackson.core:jackson-core:2.11.0
|    \--- com.fasterxml.jackson.core:jackson-annotations:2.11.0
+--- org.openapi4j:openapi-parser:1.0
|    \--- org.openapi4j:openapi-core:1.0
|         +--- com.fasterxml.jackson.core:jackson-databind:2.9.10 -> 2.11.0 (*) <2>
|         \--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.9.10 -> 2.10.2 <3>
|              \--- com.fasterxml.jackson.core:jackson-core:2.10.2 -> 2.11.0 <4>
+--- io.swagger.parser.v3:swagger-parser:2.0.20
     +--- io.swagger.parser.v3:swagger-parser-v3:2.0.20
          +--- io.swagger.core.v3:swagger-models:2.1.2
          |    \--- com.fasterxml.jackson.core:jackson-annotations:2.10.1 -> 2.11.0 <5>
          +--- io.swagger.core.v3:swagger-core:2.1.2
          |    +--- com.fasterxml.jackson.core:jackson-annotations:2.10.1 -> 2.11.0
          |    +--- com.fasterxml.jackson.core:jackson-databind:2.10.1 -> 2.11.0 (*)
          |    +--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.1 -> 2.10.2 (*) <6>
          |    +--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.10.1 <7>
          |         +--- com.fasterxml.jackson.core:jackson-annotations:2.10.1 -> 2.11.0
          |         +--- com.fasterxml.jackson.core:jackson-core:2.10.1 -> 2.11.0
          |         \--- com.fasterxml.jackson.core:jackson-databind:2.10.1 -> 2.11.0 (*)
          +--- com.fasterxml.jackson.core:jackson-annotations:2.10.2 -> 2.11.0
          +--- com.fasterxml.jackson.core:jackson-databind:2.10.2 -> 2.11.0 (*)
          \--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.2 (*) <8>
Enter fullscreen mode Exit fullscreen mode

There is a bit of a mess regarding the jackson packages and their versions. Here are a couple of entries:

<1> some with version 2.11.0
<2> upgrade from 2.9.10 to 2.11.0
<3> upgrade from 2.9.10 to 2.10.2
<4> upgrade from 2.10.2 to 2.11.0
<5> upgrade from 2.10.1 to 2.11.0
<6> upgrade from 2.10.1 to 2.10.2
<7> version 2.10.1
<8> version 2.10.2

The 3 top-level dependencies are direct dependencies (as gradle calls them) of my project and all of them use different jackson versions:

. com.fasterxml.jackson.module:jackson-module-kotlin:2.11.0
. org.openapi4j:openapi-parser:1.0
. io.swagger.parser.v3:swagger-parser:2.0.20

The first one is simple. Because com.fasterxml.jackson.module:jackson-module-kotlin is a jackson package itself, gradle selects 2.11.0 for the other transitive jackson dependencies.

jackson-module-kotlin (simplified)

    +--- com.fasterxml.jackson.module:jackson-module-kotlin:2.11.0
         +--- com.fasterxml.jackson.core:jackson-annotations:2.11.0
         +--- com.fasterxml.jackson.core:jackson-core:2.11.0
         \--- com.fasterxml.jackson.core:jackson-databind:2.11.0
Enter fullscreen mode Exit fullscreen mode

Next one is org.openapi4j:openapi-parser:1.0. This one depends on jackson 2.9.10 and gradle upgrades everything to 2.11.0 because that is the newest version it found.

openapi-parser (simplified)

+--- org.openapi4j:openapi-parser:1.0
     +--- com.fasterxml.jackson.core:jackson-core:2.10.2 -> 2.11.0
     +--- com.fasterxml.jackson.core:jackson-databind:2.9.10 -> 2.11.0 (*)
     \--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.9.10 -> 2.10.2
Enter fullscreen mode Exit fullscreen mode

Ok, not for all jackson packages. It doesn't use 2.11.0 for jackson-dataformat-yaml.

It is not using 2.11.0 because gradle has not seen a newer version of jackson-dataformat-yamlthan 2.9.10. So why is it not 2.9.10? It is not because the third direct dependency io.swagger.parser.v3:swagger-parser:2.0.20 is using 2.10.1 and 2.10.2.

swagger-parser (simplified)

+--- io.swagger.parser.v3:swagger-parser:2.0.20
     +--- com.fasterxml.jackson.core:jackson-annotations:2.10.1 -> 2.11.0
     +--- com.fasterxml.jackson.core:jackson-core:2.10.1 -> 2.11.0
     +--- com.fasterxml.jackson.core:jackson-databind:2.10.1 -> 2.11.0
     +--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.1 -> 2.10.2
     \--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.10.1
Enter fullscreen mode Exit fullscreen mode

2.10.2 for jackson-dataformat-yaml. This is the highest version for this package so gradle picks it.

Gradle finally selects the following versions:

package (stripped com.fasterxml.jackson.) version
core:jackson-annotations 2.11.0
core:jackson-core 2.11.0
core:jackson-databind 2.11.0
dataformat:jackson-dataformat-yaml 2.10.2 => want 2.11.0
datatype:jackson-datatype-jsr310 2.10.1 => want 2.11.0

Gradle uses the highest version it does find for each package. It doesn't know that the jackson packages belong together and that all of them should use the same version.

There are two possible solutions to make all jackson packages use the expected version.

solution

According to the gradle documentation the preferred solution is to add a constraints block to the dependencies. Adding a direct dependency is not the correct solution:

gradle-constraints

Often developers incorrectly fix transitive dependency issues by adding direct dependencies. To avoid this, Gradle provides the concept of dependency constraints.

Modifying the dependency resolution rules is not an option either:

gradle dependency resolve rules

If you are authoring a library, you should always prefer dependency constraints as they are published for your consumers.

constraints

The project is a library and the constraints solution look like this in the build.gradle:

ext {
    jacksonVersion = '2.11.0'
}

dependencies {
    // .. dependencies ..

    constraints {
        implementation("com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:$jacksonVersion") {
            because 'use the same version for all jackson packages'
        }
        implementation("com.fasterxml.jackson.datatype:jackson-datatype-jsr310:$jacksonVersion") {
            because 'use the same version for all jackson packages'
        }
    }

}
Enter fullscreen mode Exit fullscreen mode

Ok, this fixes the issue. :-)

Too bad each single package requires its own constraint (the closure with because ist not required). It would be nice to use wildcards to match all jackson packages with a single constraint:

ext {
    jacksonVersion = '2.11.0'
}

dependencies {
    // .. dependencies ..

    constraints {
        // does NOT work
        implementation("com.fasterxml.jackson.*:*:$jacksonVersion") {
            because 'use the same version for all jackson packages'
        }
    }

}
Enter fullscreen mode Exit fullscreen mode

That would be a bit simpler but unfortunately it does not work.

platform

I found another solution in the gradle documentation that even uses jackson as an example:

gradle dependency version alignment.

First step is to create a metadata rule that joins all jackson packages into a platform. Jackson has a platform bom that lists all packages belonging to a platform version, and it is used to create the platform rule:

class JacksonPlatformRule implements ComponentMetadataRule {
    void execute (ComponentMetadataContext ctx) {
        ctx.details.with {
            if (id.group.startsWith ("com.fasterxml.jackson")) {
                belongsTo ("com.fasterxml.jackson:jackson-bom:${id.version}", false)
            }
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

I simply added the class at the end of the build.gradle for testing, but it will probably move to gradles buildSrc folder.

Second step is to activate the rule in the dependencies block:

dependencies {
    components.all(JacksonPlatformRule)

    // .. dependencies ..
}
Enter fullscreen mode Exit fullscreen mode

By grouping the packages gradle now selects the jackson platform with the highest version and uses the versions listed in the bom for any jackson package.

This is also visible in the output of gradle dependencies (starting at line 4). Gradle selects the 2.11.0 platform module and upgrades all jackson packages to the platform version:

gradle dependencies (simplified)

+--- com.fasterxml.jackson.module:jackson-module-kotlin:2.11.0
|    +--- com.fasterxml.jackson.core:jackson-databind:2.11.0
|    |    +--- com.fasterxml.jackson.core:jackson-annotations:2.11.0
|    |    |    \--- com.fasterxml.jackson:jackson-bom:2.11.0
|    |    |         +--- com.fasterxml.jackson:jackson-bom:2.11.0 (*)
|    |    |         +--- com.fasterxml.jackson.core:jackson-annotations:2.11.0 (c)
|    |    |         +--- com.fasterxml.jackson.core:jackson-core:2.11.0 (c)
|    |    |         +--- com.fasterxml.jackson.core:jackson-databind:2.11.0 (c)
|    |    |         +--- com.fasterxml.jackson.module:jackson-module-kotlin:2.11.0 (c)
|    |    |         +--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.11.0 (c)
|    |    |         \--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.11.0 (c)
|    |    +--- com.fasterxml.jackson.core:jackson-core:2.11.0
|    |    |    \--- com.fasterxml.jackson:jackson-bom:2.11.0 (*)
|    |    \--- com.fasterxml.jackson:jackson-bom:2.11.0 (*)
|    +--- com.fasterxml.jackson.core:jackson-annotations:2.11.0 (*)
|    \--- com.fasterxml.jackson:jackson-bom:2.11.0 (*)
+--- org.openapi4j:openapi-parser:1.0
|    \--- org.openapi4j:openapi-core:1.0
|         +--- com.fasterxml.jackson.core:jackson-databind:2.9.10 -> 2.11.0 (*)
|         \--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.9.10 -> 2.11.0
|              +--- com.fasterxml.jackson.core:jackson-databind:2.11.0 (*)
|              +--- org.yaml:snakeyaml:1.26
|              +--- com.fasterxml.jackson.core:jackson-core:2.11.0 (*)
|              \--- com.fasterxml.jackson:jackson-bom:2.11.0 (*)
+--- io.swagger.parser.v3:swagger-parser:2.0.20
     +--- io.swagger.parser.v3:swagger-parser-v3:2.0.20
          +--- io.swagger.core.v3:swagger-models:2.1.2
          |    \--- com.fasterxml.jackson.core:jackson-annotations:2.10.1 -> 2.11.0 (*)
          +--- io.swagger.core.v3:swagger-core:2.1.2
          |    +--- com.fasterxml.jackson.core:jackson-annotations:2.10.1 -> 2.11.0 (*)
          |    +--- com.fasterxml.jackson.core:jackson-databind:2.10.1 -> 2.11.0 (*)
          |    +--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.1 -> 2.11.0 (*)
          |    \--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.10.1 -> 2.11.0
          |         +--- com.fasterxml.jackson.core:jackson-annotations:2.11.0 (*)
          |         +--- com.fasterxml.jackson.core:jackson-core:2.11.0 (*)
          |         +--- com.fasterxml.jackson.core:jackson-databind:2.11.0 (*)
          |         \--- com.fasterxml.jackson:jackson-bom:2.11.0 (*)
          +--- com.fasterxml.jackson.core:jackson-annotations:2.10.2 -> 2.11.0 (*)
          +--- com.fasterxml.jackson.core:jackson-databind:2.10.2 -> 2.11.0 (*)
          \--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.2 -> 2.11.0 (*)
Enter fullscreen mode Exit fullscreen mode

conclusion

The constraints solution is easy to use on single dependencies. The platform rule is the way to go for groups of packages like jackson.

That's it.

Top comments (0)

Read next

nivekalara237 profile image

Building a Spring Boot Consumer Application with Amazon SQS: Setup Infrastructure Using Cloud Development Kit (CDK)

Kevin Lactio Kemta -

arshisaxena26 profile image

Understanding Garbage Collection in Java: Essential for Interview Preparation

Arshi Saxena -

arshisaxena26 profile image

Navigating JVM Memory: Key Concepts for Your Java Interview

Arshi Saxena -

realnamehidden1_61 profile image

Find the Duplicate Elements in an Array/List

realNameHidden -