In an ever-increasingly complex world that we are living in, our expectation also rises exponentially in terms of how we demand information to be instantaneously ready for us as we are searching for them. In the computing world, there is an increasing need to handle the processing of huge volumes of data in the most efficient manner, and this is when the event-driven approach in messaging and sysem design has come to the forefront, when cloud and cloud-native computing have taken he world by storm. So can event messages become public "consumables" like what REST APIs have been since the last decade?
We'll take a look and step through a basic example of a multi- and cloud scenario, in which we'll leverage on the Kafka message broker running in a managed cloud platform - Confluent, and is responsible for producing messages to a topic, from which the Kafka consumer client as enabled by MicroProfile Reactive Messaging's Kafka Connector, and encapsulated in the Open Liberty runtime which gets powered underneath by the very efficient IBM Semeru Runtimes - the OpenJ9 JVM, and the client is launched using OpenShift onto the IBM Cloud. On the producer side, the Confluent Cloud can be launched on one of their support external cloud providers - Microsoft Azure, Amazon AWS, or the Google Cloud Platform.
Let's dive in:
Message producing side is handled by the Confluent Cloud. The details will not be discussed here, as this article focuses primarily on the message consumer side. Please refer to the documentations on the Confluent website, such as How to build your first Apache KafkaProducer application. Another way, a quicker way, is to leverage on the Confluent Cloud console to create a topic.
Accessing your OpenShift cluster (please see one of the following ways accordingly):
- (FOR WORKSHOP PARTICIPANTS) If you are given a temporary OpenShift cluster at an IBM workshop: IBM Digital Developer Conference - Hybrid Cloud 2021 Please refer to the instructions that have been provided to you as a Bookmark URL in the #ddc-labs-courses Slack channel.
- (FOR CUSTOMERS/TRIAL CUSTOMERS) If you are ready to create a new OpenShift cluster, please refer to the IBM Cloud documentation on how to create your Red Hat OpenShift cluster.
* OpenShift console (via the IBM Cloud console) * OpenShift CLI (`oc`) * IBM Cloud shell (via the IBM Cloud console)
Once you are logged on to your OpenShift cluster, you will need to install the Open Liberty operator. This will help us to deploy and manage our Open Liberty microservices application on the cluster. You can do so via the OpenShift console, or on the command-line.
Once the Open Liberty operator has been installed, you may want to verify that has succeeded.
oc api-resources --api-group=openliberty.io
Add a private Docker credential:
Docker limits container image pull requests for free DockerHub subscriptions. For more information, see Understanding Docker Hub Rate Limiting. If you have a Docker account with a Pro or Team subscription, you can add a private credential to avoid any errors as a result of the limits.
To add a private credential, navigate to the OpenShift web console and select Workloads > Secrets from the sidebar menu. Ensure that the selected project is openshift-config. Search for pull-secret and click the three vertical dots menu; then Edit Secret. Scroll down and click Add credentials. Enter docker.io to the Registry Server Address field, your Docker user ID to the Username field, and your Docker password to the Password field. Click the Save button to save the credential.
Use the example microservice (and currently we have named it as inventory) by cloning from the following repository:
git clone https://github.com/mgrygles-lab/guide-ddc.git
To deploy the inventory microservice, you must first package it, then create and run an OpenShift build to produce runnable container image of the packaged microservice.
Packaging the microservice
Ensure that you are in the start directory and run the following commands to package the system and inventory microservices:
mvn clean install mvn clean package
Building and pushing the images
Require: a build template to configure how to build your container images. We have provided the build template for you.
Examine the build.yaml template file.
The build.yaml template includes two objects. The ImageStream object provides an abstraction from the image in the image registry. This allows you to reference and tag the image. The image registry used is the integrated internal OpenShift Container Registry.
The BuildConfig object defines a single build definition and any triggers that kickstart the build. The source spec defines the build input. In this case, the build inputs are your binary (local) files, which are streamed to OpenShift for the build. The uploaded files need to include the packaged WAR application binaries, which is why you needed to run the Maven commands. The template specifies a Docker strategy build, which invokes the docker build command, and creates a runnable container image of the microservice from the build input. The template is parameterized with the APP_NAME parameter so that you can use the same template to create the objects for the system and inventory microservices separately.
Run the following commands to create the objects for the system and inventory microservices:
oc process -f build.yaml -p APP_NAME=inventory | oc create -f -
Next, run the following commands to view the newly created ImageStream object and the build configuration for the microservice:
oc get all -l name=inventory
Look for the following resources similar to the following:
|imagestream.image.openshift.io/inventory-imagestream||image-registry.openshift-image-registry.svc:5000/guide/inventory-imagestream||1.0-SNAPSHOT||2 days ago|
Ensure that you are in the ddc directory and trigger the builds by running the following commands:
oc start-build inventory-buildconfig --from-dir=inventory/.
The local inventory directory is uploaded to OpenShift to be built into the Docker image. Run the following command to list the builds and track their status:
oc get builds
Look for the output that is similar to the following example:
|inventory-buildconfig-1||Docker||Binary@f24cb58||Running||13 seconds ago|
You may need to wait some time until the build is complete. To check whether the build is complete, run the following commands to view the build logs until the Push successful message appears:
oc logs build/inventory-buildconfig-1
Run the following command to view the ImageStream objects:
oc get imagestreams
Run the following commands to get more details on the newly pushed images within the streams:
oc describe imagestream/inventory-imagestream
The following example shows part of the inventory-imagestream output:
Created: 2 minutes ago
Image Repository: image-registry.openshift-image-registry.svc:5000/guide/inventory-imagestream
Image Lookup: local=false
Unique Images: 1
Now you’re ready to deploy the images.
You can configure the specifics of the Open Liberty Operator-controlled deployment with a YAML configuration file.
Examine the deploy.yaml configuration file.
The deploy.yaml file is configured to deploy the OpenLibertyApplication resource, named inventory, which is controlled by the Open Liberty Operator.
The applicationImage parameter defines what container image is deployed as part of the OpenLibertyApplication CRD. This parameter follows the /[:tag] format. The parameter can also point to an image hosted on an external registry, such as Docker Hub.
One of the benefits of using the ImageStream object is that the operator redeploys the application when it detects a new image is pushed. The env parameter is used to specify environment variables that are passed to the container at runtime. You need to specify the bootstrap address required to communicate with the Kafka cluster as an environment variable.
Update the [kafka-bootstrap-address] variable to the bootstrap address of the remote Kafka cluster that's hosting the producer, such as:
Run the following command to deploy the inventory microservice with the previously explained configurations:
oc apply -f deploy.yaml
Next, run the following command to view your newly created OpenLibertyApplications resource:
oc get OpenLibertyApplications
You can also replace OpenLibertyApplications with the shortname olapps.
Look for output that is similar to the following example:
A RECONCILED state value of True indicates that the operator was able to successfully process the OpenLibertyApplications instance. Run the following commands to view details of your microservice:
oc describe olapps/inventory
This example shows part of the olapps/inventory output:
API Version: openliberty.io/v1beta1