Config Gradle to generate Java code from Protobuf

In the last lecture, we have finished writing our protocol buffer messages and generate Go codes from them. Today we're going to do the same for Java.

Here's the link to the full gRPC course playlist on Youtube
Github repository: pcbook-go and pcbook-java
Gitlab repository: pcbook-go and pcbook-java

We will create a new Gradle project, setup some plugins and config them to automatically generate Java code whenever the project is built. We will also learn how to use option to customise the generated codes.

OK, let's start!

New Gradle project

First, make sure you already have Java Development Kit and IntelliJ IDEA installed on your computer.

Open IntelliJ IDEA and create a new project. We will use Gradle because Google has an official Gradle plugin for protocol buffer. For the project SDK, make sure that the correct Java version is selected. It should be at least Java 8. Then click Next.

Fill in the GroupID, the ArtifactID. Check the project name and its location to be exactly what you want. Then click Finish. It might take a few seconds for IntelliJ IDEA to setup the project.

Config protobuf-gradle plugin

Now we will open the build.gradle file to setup some plugins. The first one is protobuf-gradle plugins from Google.

Open this Github page, scroll down a bit, copy the plugins block and paste it to our build.gradle file:

plugins {
    id "com.google.protobuf" version "0.8.10"
    id "java"
}

group 'com.gitlab.techschool'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'
}

Add protobuf-java dependency

Next, we need to add a dependency: protobuf-java to our project. Let's open this maven repository page.

It's the protobuf-java artifact of com.google.protobuf package. Let's select the latest version: 3.10.0

Click on the Gradle tab and copy the setting, then paste it into the dependencies block of our build.gradle file:

plugins {
    id "com.google.protobuf" version "0.8.10"
    id "java"
}

group 'com.gitlab.techschool'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'

    // https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java
    compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.10.0'
}

Add grpc-all dependency

We will need a package to work with gRPC as well. So let's go back to the maven repository and search for grpc-all. It should be found at this page.

Select the latest version (1.25.0 in my case). Similar as before, open the Gradle tab and copy the setting, then paste it to the dependencies block of our build.gradle file. IntelliJ IDEA will automatically detect and configure it for us.

plugins {
    id "com.google.protobuf" version "0.8.10"
    id "java"
}

group 'com.gitlab.techschool'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'

    // https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java
    compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.10.0'

    // https://mvnrepository.com/artifact/io.grpc/grpc-all
    compile group: 'io.grpc', name: 'grpc-all', version: '1.25.0'

}

Setup protobuf compiler

Now we will setup protobuf compiler. By default, the protobuf-gradle-plugin will search for protoc executable in the system. We've already installed it in previous lecture with Homebrew. However, if you come here directly for Java, I will show you another way to get the pre-compiled protoc.

First, go to the maven repository and look for protoc. It can be found in this page.

Select the latest version (3.10.1 in my case), but don't add it to the dependencies block as before. Instead, we will config it in a separate block.

Let's go back to the protobuf-gradle-plugin github page, and copy this setting block:

Paste it in our build.gradle file, replace the version with the latest one that we've found above, and IntelliJ will take care of the rest.

plugins {
    id "com.google.protobuf" version "0.8.10"
    id "java"
}

group 'com.gitlab.techschool'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'

    // https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java
    compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.10.0'

    // https://mvnrepository.com/artifact/io.grpc/grpc-all
    compile group: 'io.grpc', name: 'grpc-all', version: '1.25.0'
}

protobuf {
    protoc {
        artifact = 'com.google.protobuf:protoc:3.10.1'
    }
}

Tell protoc to use gRPC plugin

Now, a very important thing that we should do is to tell protoc to use the gRPC plugin when generating Java code. We can do that by getting back to the github page of protobuf-gradle-plugin and look for this plugins block.

There's an artifact for protoc-gen-grpc-java, we can search for its latest version on the maven repository.

It's 1.25.0 in my case, so let's copy & paste the config block to build.gradle file, and change the version to 1.25.0:

plugins {
    id "com.google.protobuf" version "0.8.10"
    id "java"
}

group 'com.gitlab.techschool'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'

    // https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java
    compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.10.0'

    // https://mvnrepository.com/artifact/io.grpc/grpc-all
    compile group: 'io.grpc', name: 'grpc-all', version: '1.25.0'
}

protobuf {
    protoc {
        artifact = 'com.google.protobuf:protoc:3.10.1'
    }

    plugins {
        grpc {
            artifact = 'io.grpc:protoc-gen-grpc-java:1.25.0'
        }
    }

    generateProtoTasks {
        all()*.plugins {
            grpc {}
        }
    }
}

We should also add the generateProtoTasks settings to guide protoc to use the gRPC plugin.

I know it looks complicated and there are too many stuffs, but it will help us speed up a lot in the development phase.

Where the generated codes are located

There's one more thing left before we're good to go, that's to tell IntelliJ IDEA where our generated codes will be located. That way it can easily and correctly do code analysis and code suggestion for us later.

To do that, we use the sourceSets block. And inside this block, we specify 2 source directories: 1 for gRPC, and the other for normal protobuf messages:

plugins {
    id "com.google.protobuf" version "0.8.10"
    id "java"
}

group 'com.gitlab.techschool'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'

    // https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java
    compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.10.0'

    // https://mvnrepository.com/artifact/io.grpc/grpc-all
    compile group: 'io.grpc', name: 'grpc-all', version: '1.25.0'
}

sourceSets {
    main {
        java {
            srcDirs 'build/generated/source/proto/main/grpc'
            srcDirs 'build/generated/source/proto/main/java'
        }
    }
}

protobuf {
    protoc {
        artifact = 'com.google.protobuf:protoc:3.10.1'
    }

    plugins {
        grpc {
            artifact = 'io.grpc:protoc-gen-grpc-java:1.25.0'
        }
    }

    generateProtoTasks {
        all()*.plugins {
            grpc {}
        }
    }
}

OK, now we are really good to go!

Generate Java codes

Let's create a new proto folder inside src/main. Then copy all the proto files that we have written in previous lectures to this folder.

Now comes the interesting part. Once we click the Build icon, IntelliJ IDEA will start some background tasks to generate codes for us. When they finished, we can find the generated codes in this folder:

build/generated/source/proto/main/java

The grpc folder is empty now because we haven't written any RPC yet. In the java folder, there are 6 Java files, one for each of the messages.

There are a lot of codes in 1 single file. And the generated package name is the same as the protobuf package: techschool.pcbook. We can easily change this package name similar to what we did for Go by this setting:

option java_package="com.gitlab.techschool.pcbook.pb"

We can also tell protoc to split the codes into smaller files instead of putting them inside 1 single big file. It's pretty simple, just set:

option java_multiple_files = true;

These options should be added to all proto files. For example, the laptop_message.proto file will be like this:

syntax = "proto3";

package techschool.pcbook;

option go_package = "pb";
option java_package = "com.gitlab.techschool.pcbook.pb";
option java_multiple_files = true;

import "processor_message.proto";
import "memory_message.proto";
import "storage_message.proto";
import "screen_message.proto";
import "keyboard_message.proto";
import "google/protobuf/timestamp.proto";

message Laptop {
    string id = 1;
    string brand = 2;
    string name = 3;
    CPU cpu = 4;
    Memory ram = 5;
    repeated GPU gpus = 6;
    repeated Storage storages = 7;
    Screen screen = 8;
    Keyboard keyboard = 9;
    oneof weight {
        double weight_kg = 10;
        double weight_lb = 11;
    }
    double price_usd = 12;
    uint32 release_year = 13;
    google.protobuf.Timestamp updated_at = 14;
}

Now if I delete the old generated codes and rebuild the project, there will be many java files, and the package name will be changed to what we want.

And that's it! We're done!

In the next lecture, we will start writing codes in Go and Java to serialise protobuf messages to binary and JSON.

Thank you for reading and see you then!

If you like the article, please subscribe to our Youtube channel and follow us on Twitter for more tutorials in the future.

If you want to join me on my current amazing team at Voodoo, check out our job openings here. Remote or onsite in Paris/Amsterdam/London/Berlin/Barcelona with visa sponsorship.