In the previous article, we looked at one way to create a custom operator that manages the IRIS instance state. This time, we’re going to take a look at a ready-to-go operator, InterSystems Kubernetes Operator (IKO). Official documentation will help us navigate the deployment steps.
Prerequisites
To deploy IRIS, we need a Kubernetes cluster. In this example, we’ll use Google Kubernetes Engine (GKE), so we’ll need to use a Google account, set up a Google Cloud project, and install gcloud and kubectl command line utilities.
You’ll also need to install the Helm3 utility:
$ helm version
version.BuildInfo{Version:"v3.3.4"...}
Note: Be aware that on Google free tier, not all resources are free.
It doesn’t matter in our case which type of GKE we use – zonal, regional, or private. After we create one, let’s connect to the cluster. We’ve created a cluster called "iko" in a project called "iko-project". Use your own project name in place of "iko-project" in the later text.
This command adds this cluster to our local clusters configuration:
$ gcloud container clusters get-credentials iko --zone europe-west2-b --project iko-project
Install IKO
Let’s deploy IKO into our newly-created cluster. The recommended way to install packages to Kubernetes is using Helm. IKO is not an exception and can be installed as a Helm chart. Choose Helm version 3 as it's more secure.
Download IKO from the WRC page InterSystems Components, creating a free developer account if you do not already have one. At the moment of writing, the latest version is 2.0.223.0.
Download the archive and unpack it. We will refer to the unpacked directory as the current directory.
The chart is in the chart/iris-operator directory. If you just deploy this chart, you will receive an error when describing deployed pods:
Failed to pull image "intersystems/iris-operator:2.0.0.223.0": rpc error:
code = Unknown desc = Error response from daemon:
pull access denied for intersystems/iris-operator,
repository does not exist or may require 'docker login'.
So, you need to make an IKO image available from the Kubernetes cluster. Let’s push this image into Google Container Registry first:
$ docker load -i image/iris_operator-2.0.0.223.0-docker.tgz
$ docker tag intersystems/iris-operator:2.0.0.223.0 eu.gcr.io/iko-project/iris-operator:2.0.0.223.0
$ docker push eu.gcr.io/iko-project/iris-operator:2.0.0.223.0
After that, we need to direct IKO to use this new image. You should do this by editing the Helm values file:
$ vi chart/iris-operator/values.yaml
...
operator:
registry: eu.gcr.io/iko-project
...
Now, we’re ready to deploy IKO into GKE:
$ helm upgrade iko chart/iris-operator --install --namespace iko --create-namespace
$ helm ls --all-namespaces --output json | jq '.[].status'
"deployed"
$ kubectl -n iko get pods # Should be Running with Readiness 1/1
Let’s look at the IKO logs:
$ kubectl -n iko logs -f --tail 100 -l app=iris-operator
…
I1212 17:10:38.119363 1 secure_serving.go:116] Serving securely on [::]:8443
I1212 17:10:38.122306 1 operator.go:77] Starting Iris operator
Custom Resource Definition irisclusters.intersystems.com was created during IKO deployment.
You can look at the API schema it supports, although it is quite long:
$ kubectl get crd irisclusters.intersystems.com -oyaml | less
One way to look at all available parameters is to use the "explain" command:
$ kubectl explain irisclusters.intersystems.com
Another way is using jq. For instance, viewing all top-level configuration settings:
$ kubectl get crd irisclusters.intersystems.com -ojson | jq '.spec.versions[].schema.openAPIV3Schema.properties.spec.properties | to_entries[] | .key'
"configSource"
"licenseKeySecret"
"passwordHash"
"serviceTemplate"
"topology"
Using jq in this way (viewing the configuration fields and their properties), we can find out the following configuration structure:
configSource
name
licenseKeySecret
name
passwordHash
serviceTemplate
metadata
annotations
spec
clusterIP
externalIPs
externalTrafficPolicy
healthCheckNodePort
loadBalancerIP
loadBalancerSourceRanges
ports
type
topology
arbiter
image
podTemplate
controller
annotations
metadata
annotations
spec
affinity
nodeAffinity
preferredDuringSchedulingIgnoredDuringExecution
requiredDuringSchedulingIgnoredDuringExecution
podAffinity
preferredDuringSchedulingIgnoredDuringExecution
requiredDuringSchedulingIgnoredDuringExecution
podAntiAffinity
preferredDuringSchedulingIgnoredDuringExecution
requiredDuringSchedulingIgnoredDuringExecution
args
env
imagePullSecrets
initContainers
lifecycle
livenessProbe
nodeSelector
priority
priorityClassName
readinessProbe
resources
schedulerName
securityContext
serviceAccountName
tolerations
preferredZones
updateStrategy
rollingUpdate
type
compute
image
podTemplate
controller
annotations
metadata
annotations
spec
affinity
nodeAffinity
preferredDuringSchedulingIgnoredDuringExecution
requiredDuringSchedulingIgnoredDuringExecution
podAffinity
preferredDuringSchedulingIgnoredDuringExecution
requiredDuringSchedulingIgnoredDuringExecution
podAntiAffinity
preferredDuringSchedulingIgnoredDuringExecution
requiredDuringSchedulingIgnoredDuringExecution
args
env
imagePullSecrets
initContainers
lifecycle
livenessProbe
nodeSelector
priority
priorityClassName
readinessProbe
resources
limits
requests
schedulerName
securityContext
serviceAccountName
tolerations
preferredZones
replicas
storage
accessModes
dataSource
apiGroup
kind
name
resources
limits
requests
selector
storageClassName
volumeMode
volumeName
updateStrategy
rollingUpdate
type
data
image
mirrored
podTemplate
controller
annotations
metadata
annotations
spec
affinity
nodeAffinity
preferredDuringSchedulingIgnoredDuringExecution
requiredDuringSchedulingIgnoredDuringExecution
podAffinity
preferredDuringSchedulingIgnoredDuringExecution
requiredDuringSchedulingIgnoredDuringExecution
podAntiAffinity
preferredDuringSchedulingIgnoredDuringExecution
requiredDuringSchedulingIgnoredDuringExecution
args
env
imagePullSecrets
initContainers
lifecycle
livenessProbe
nodeSelector
priority
priorityClassName
readinessProbe
resources
limits
requests
schedulerName
securityContext
serviceAccountName
tolerations
preferredZones
shards
storage
accessModes
dataSource
apiGroup
kind
name
resources
limits
requests
selector
storageClassName
volumeMode
volumeName
updateStrategy
rollingUpdate
type
There are so many settings, but, you don’t need to set them all. The defaults are suitable. You can see examples of configuration in the file iris_operator-2.0.0.223.0/samples.
To run a minimal viable IRIS, we need to specify only a few settings, like IRIS (or IRIS-based application) version, storage size, and license key.
Note about license key: we’ll use a community IRIS, so we don’t need a key. We cannot just omit this setting, but can create a secret containing a pseudo-license. License secret generation is simple:
$ touch iris.key # remember that a real license file is used in the most cases
$ kubectl create secret generic iris-license --from-file=iris.key
An IRIS description understandable by IKO is:
$ cat iko.yaml
apiVersion: intersystems.com/v1alpha1
kind: IrisCluster
metadata:
name: iko-test
spec:
passwordHash: '' # use a default password SYS
licenseKeySecret:
name: iris-license # use a Secret name bolded above
topology:
data:
image: intersystemsdc/iris-community:2020.4.0.524.0-zpm # Take a community IRIS
storage:
resources:
requests:
storage: 10Gi
Send this manifest into the cluster:
$ kubectl apply -f iko.yaml
$ kubectl get iriscluster
NAME DATA COMPUTE MIRRORED STATUS AGE
iko-test 1 Creating 76s
$ kubectl -n iko logs -f --tail 100 -l app=iris-operator
db.Spec.Topology.Data.Shards = 0
I1219 15:55:57.989032 1 iriscluster.go:39] Sync/Add/Update for IrisCluster default/iko-test
I1219 15:55:58.016618 1 service.go:19] Creating Service default/iris-svc.
I1219 15:55:58.051228 1 service.go:19] Creating Service default/iko-test.
I1219 15:55:58.216363 1 statefulset.go:22] Creating StatefulSet default/iko-test-data.
We see that some resources (Service, StatefulSet) are going to be created in a cluster in the "default" namespace.
In a few seconds, you should see an IRIS pod in the "default" namespace:
$ kubectl get po -w
NAME READY STATUS RESTARTS AGE
iko-test-data-0 0/1 ContainerCreating 0 2m10s
Wait a little until the IRIS image is pulled, that is, until Status becomes Ready and Ready becomes 1/1. You can check what type of disk was created:
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-b356a943-219e-4685-9140-d911dea4c106 10Gi RWO Delete Bound default/iris-data-iko-test-data-0 standard 5m
Reclaim policy "Delete" means that when you remove Persistent Volume, GCE persistent disk will be also removed. There is another policy, "Retain", that allows you to save Google persistent disks to survive Kubernetes Persistent Volumes deletion. You can define a custom StorageClass to use this policy and other non-default settings. An example is present in IKO’s documentation: Create a storage class for persistent storage.
Now, let’s check our newly created IRIS. In general, traffic to pods goes through Services or Ingresses. By default, IKO creates a service of ClusterIP type with a name from the iko.yaml metadata.name field:
$ kubectl get svc iko-test
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
iko-test ClusterIP 10.40.6.33 <none> 1972/TCP,52773/TCP 14m
We can call this service using port-forward:
$ kubectl port-forward svc/iko-test 52773
Navigate a browser to http://localhost:52773/csp/sys/UtilHome.csp and type _system/SYS.
You should see a familiar IRIS user interface (UI).
Custom Application
Let’s replace a pure IRIS with an IRIS-based application. First, download the COVID-19 application. We won’t consider a complete, continuous deployment here, just minimal steps:
$ git clone https://github.com/intersystems-community/covid-19.git
$ cd covid-19
$ docker build --no-cache -t covid-19:v1 .
As our Kubernetes is running in a Google cloud, let’s use Google Docker Container Registry as an image storage. We assume here that you have an account in Google Cloud allowing you to push images. Use your own project name in the below-mentioned commands:
$ docker tag covid-19:v1 eu.gcr.io/iko-project/covid-19:v1
$ docker push eu.gcr.io/iko-project/covid-19:v1
Let’s go to the directory with iko.yaml, change the image there, and redeploy it. You should consider removing the previous example first:
$ cat iko.yaml
...
data:
image: eu.gcr.io/iko-project/covid-19:v1
...
$ kubectl delete -f iko.yaml
$ kubectl -n iko delete deploy -l app=iris-operator
$ kubectl delete pvc iris-data-iko-test-data-0
$ kubectl apply -f iko.yaml
You should recreate the IRIS pod with this new image.
This time, let’s provide external access via Ingress Resource. To make it work, we should deploy an Ingress Controller (choose nginx for its flexibility). To provide a traffic encryption (TLS), we will also add yet another component – cert-manager.
To install both these components, we use a Helm tool, version 3.
$ helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
$ helm upgrade nginx-ingress \
--namespace nginx-ingress \
ingress-nginx/ingress-nginx \
--install \
--atomic \
--version 3.7.0 \
--create-namespace
Look at an nginx service IP (it’s dynamic, but you can make it static):
$ kubectl -n nginx-ingress get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-ingress-ingress-nginx-controller LoadBalancer 10.40.0.103 xx.xx.xx.xx 80:32032/TCP,443:32374/TCP 88s
Note: your IP will differ.
Go to your domain registrar and create a domain name for this IP. For instance, create an A-record:
covid19.myardyas.club = xx.xx.xx.xx
Some time will pass until this new record propagates across DNS servers. The end result should be similar to:
$ dig +short covid19.myardyas.club
xx.xx.xx.xx
Having deployed Ingress Controller, we now need to create an Ingress resource itself (use your own domain name):
$ cat ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: iko-test
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
certmanager.k8s.io/cluster-issuer: lets-encrypt-production # Cert manager will be deployed below
spec:
rules:
- host: covid19.myardyas.club
http:
paths:
- backend:
serviceName: iko-test
servicePort: 52773
path: /
tls:
- hosts:
- covid19.myardyas.club
secretName: covid19.myardyas.club
$ kubectl apply -f ingress.yaml
After a minute or so, IRIS should be available at http://covid19.myardyas.club/csp/sys/UtilHome.csp (remember to use your domain name) and the COVID-19 application at http://covid19.myardyas.club/dsw/index.html (choose namespace IRISAPP).
Note: Above, we’ve exposed the HTTP IRIS port. If you need to expose via nginx TCP super-server port (1972 or 51773), you can read instructions at Exposing TCP and UDP services.
Add Traffic Encryption
The last step is to add traffic encryption. Let’s deploy cert-manager for that:
$ kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/v0.10.0/deploy/manifests/00-crds.yaml
$ helm upgrade cert-manager \
--namespace cert-manager \
jetstack/cert-manager \
--install \
--atomic \
--version v0.10.0 \
--create-namespace
$ cat lets-encrypt-production.yaml
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
name: lets-encrypt-production
spec:
acme:
# Set your email. Let’s Encrypt will send notifications about certificates expiration
email: mvhoma@gmail.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: lets-encrypt-production
solvers:
- http01:
ingress:
class: nginx
$ kubectl apply -f lets-encrypt-production.yaml
Wait a few minutes until cert-manager notices IRIS-application ingress and goes to Let’s Encrypt for a certificate. You can observe Order and Certificate resources in progress:
$ kubectl get order
NAME STATE AGE
covid19.myardyas.club-3970469834 valid 52s
$ kubectl get certificate
NAME READY SECRET AGE
covid19.myardyas.club True covid19.myardyas.club 73s
This time, you can visit a more secured site version - https://covid19.myardyas.club/dsw/index.html:
About Native Google Ingress Controller and Managed Certificates
Google supports its own ingress controller, GCE, which you can use in place of an nginx controller. However, it has some drawbacks, for instance, lack of rewrite rules support, at least at the moment of writing.
Also, you can use Google managed certificates in place of cert-manager. It’s handy, but initial retrieval of certificate and any updates of Ingress resources (like new path) causes a tangible downtime. Also, Google managed certificates work only with GCE, not with nginx, as noted in Managed Certificates.
Next Steps
We’ve deployed an IRIS-based application into the GKE cluster. To expose it to the Internet, we’ve added Ingress Controller and a certification manager. We’ve tried the IrisCluster configuration to highlight that setting up IKO is simple. You can read about more settings in Using the InterSystems Kubernetes Operator documentation.
A single data server is good, but the real fun begins when we add ECP, mirroring, and monitoring, which are also available with IKO. Stay tuned and read the upcoming article in our Kubernetes operator series to take a closer look at mirroring.
Top comments (0)