Welcome once again to my side project, distracting myself from the horrors of modern life by writing about class loading on the JVM, which I think is cheaper than therapy. This third post takes us beyond the theory of Part 2 and explores a fun and interesting use-case for build maintainers.

Gradle’s class loader hierarchy

To understand the monkey patching technique we’ll discuss below, I want to start with a brief exploration of Gradle’s class loader hierarchy. Whereas the JVM has three standard class loaders (as we learned in Part 1), Gradle’s class loader hierarchy is… rather more complex.

With some quick scripting (gist), we can introspect that hierarchy, which I present below in all of its raw glory.¹

1. VisitableURLClassLoader(groovy-script-/home/tony/workspace/temp/classloaders-simple/app/build.gradle-loader)
2. VisitableURLClassLoader(ClassLoaderScopeIdentifier.Id{coreAndPlugins:settings:/home/tony/workspace/temp/classloaders-simple/buildSrc:root-project(export)})
3. VisitableURLClassLoader(ClassLoaderScopeIdentifier.Id{coreAndPlugins:settings:/home/tony/workspace/temp/classloaders-simple/buildSrc(export)})
4. VisitableURLClassLoader(ClassLoaderScopeIdentifier.Id{coreAndPlugins:settings(export)})
5. CachingClassLoader(FilteringClassLoader(VisitableURLClassLoader(legacy-mixin-loader)))
6. FilteringClassLoader(VisitableURLClassLoader(legacy-mixin-loader))
7. VisitableURLClassLoader(legacy-mixin-loader)
8. VisitableURLClassLoader(ant-and-gradle-loader)
9. VisitableURLClassLoader(ant-loader)
10. jdk.internal.loader.ClassLoaders$PlatformClassLoader@cf73672

Most of this hierarchy is effectively immutable from our perspective (and so we will ignore it), but the first four class loaders can be influenced by our build scripts. The first class loader is used by any class loaded by app/build.gradle. Its immediate parent is the root build.gradle.² You may (correctly) infer from this that the root build.gradle class loader is the parent of all subproject class loaders. Above that is buildSrc itself, which is the historical source of primordial evil in Gradleland.³ And finally, right above that and our last chance⁴ to influence the classpath in a typical Gradle build, is settings.gradle.⁵

To the obvious question — how can I misuse this information? — I present the obvious answer — why, via the source of all primordial evil, buildSrc, of course!

Monkey patching with buildSrc

“buildSrc” refers to a directory by that name in the root of the project, and which contains build logic and custom plugins used by the project. Anything placed in buildSrc is automatically on the classpath of all projects in a build, as indicated above. See the docs for more information.

As mentioned above, buildSrc defines the class loader that is the parent of the root script class loader, which is in turn the parent of all the subprojects’ class loaders. If we also recall that class loaders always delegate to their parent first, and that classpaths are order-sensitive, we can understand the monkey patching technique we're about to explore.

Imagine there is an issue in AGP that we would like to patch. Forking and publishing custom versions of AGP is complicated, so it's simpler to patch it locally while waiting for the fix to be applied upstream. Consider the following:

// buildSrc/build.gradle
dependencies {
  implementation 'com.android.tools.build:gradle:4.1.0'
}

// app/build.gradle
plugins {
  id 'com.android.application'
}

This works as expected. We don't need to specify AGP on the build script's classpath directly, and we don't even need to supply a version in the plugins block (in fact, doing so would be a Gradle error).

Now let's apply our patch. We'll add a new file to our project, the salient parts of which are included below. The full file is too large to include directly in this post, but you can see it in a gist.

// buildSrc/src/main/java/com/android/build/gradle/tasks/factory/AndroidUnitTest.java

// note the package!
package com.android.build.gradle.tasks.factory;

public abstract class AndroidUnitTest
    extends Test
    implements VariantAwareTask {

  @Override
  @TaskAction
  public void executeTests() {
    getLogger()
      .quiet("Tests passed! (I think?)");
  }
}

In our scenario, writing good tests is too hard, but we have a KPI that calls for 100% test coverage.

Now execute the unit tests in your project:

./gradlew app:testDebugUnitTest

Amidst all the other output, you should see this:

> Task :app:testDebugUnitTest
Tests passed! (I think?)

Why does this work? Leaving aside the abuse of Java's package visibility, the reason this works is because classpaths are order-dependent. Our version of AndroidUnitTest is on the build classpath before the version that is packaged with AGP (but it can still be compiled, because AGP is indeed on the classpath), and this means that the JVM runtime stops looking when it finds our class definition during build execution. Therefore it is our class that is added to this root class loader for use by all our subprojects; the class file provided by AGP is effectively ignored.

🤯

Thanks to César Puerta for telling me about this technique a couple years ago, and my old colleagues at Gradle for teaching me why it works. César used the technique for patching in support for the Gradle build cache in Android unit tests before it had been officially added to upstream AGP. In fact, he tells me they continue to use this technique, such as with Lint and the Kotlin compiler.

This technique would not work if the jar in question (AGP) were on the classpath of a parent class loader, thanks to Java’s delegation model. It works here because our custom code and AGP are loaded by the same class loader.

What's up next?

There's a lot more to explore in the build domain aaand… I may or may not get to it. If there's something you'd like to learn more about, feel free to drop a comment or hit me up on Twitter. I would very much like to finish up this series with a post on the compile and runtime classpaths for your actual application. So stay tuned!

Endnotes

¹ This was on a system running Ubuntu 20.04, using Gradle 6.8.1 and Java 11. up
² Which includes the interesting substring “buildSrc:root-project(export)”. What does this have to do with buildSrc?! I have so many questions! up
³ A joking reference to the fact that any change to buildSrc means a change to the classpath of every project in your build, which means a full re-build with every change to buildSrc. It's easy to misuse this power! up
⁴ This is obviously a gross over-simplification. Gradle also has init scripts, composite builds, the Gradle wrapper itself, and of course the JDK installed on your system. up
⁵ Pre-Gradle 6, the order of the third and fourth class loaders was inverted. up