Kubernetes-based development with Devspace
Modern applications base more and more on micro-services. Splitting large applications into smaller pieces makes the whole more maintainable and easier to develop. However, instead of developing a big monolith, we work on a bunch of tiny applications, making it more challenging to debug and deploy the whole system. Luckily, there are many tools out there to help us out. An interesting comparison of some of them can be found here. In what follows, we want to see how easy it is to do Kubernetes-based development with devspace.
A micro-services application
Suppose we are developing a micro-services application, for example an e-shop. In essence, our e-shop consists of a frontend application that communicates with a backend through an API. For the sake of simplicity, let's say that our backend looks like this:
User management is handled by the iam-service
. Orders are processed via the message-queue
. Most of our backend's business logic is packed in serverless functions served by the faas
. Our application's state is held in our database
. Finally, for some good reasons (e.g. the ease of testing setup), we are developing our software in a monorepo.
With time, our micro-services application will necessarily contain a lot of business logic that will be packed in even more micro-service code or serverless functions. For example, we might need a connector service between our message-queue
and our faas
, or an assets service with some logic to add new assets in a controlled way. A very convenient way to host our micro-services is to dockerize them and let Kubernetes orchestrate them.
Typically, our IAM service is a third-party like keycloak or fusionauth which we can easily deploy on Kubernetes by means of a helm chart. Helm is a very practical package manager for Kubernetes. For example, a typical fusionauth deployment would look like something along these lines:
helm repo add fusionauth https://fusionauth.github.io/charts
helm install fusionauth --create-namespace fusionauth/fusionauth --namespace auth \
--set database.protocol=postgresql \
--set database.user=<username> \
--set database.password=<password> \
--set database.host=<hostname> \
--set database.port=5432 \
--set database.name=<database-name> \
--set database.root.user=<root-user> \
--set database.root.password=<root-password> \
--set app.runtimeMode=production \
--set search.engine=database
Our message queue is probably redismq, rabbitmq or kubemq, for which we also easily find helm charts.
Then come our own custom services for which we need to write our own Kubernetes resources (deployments, services, ingresses, etc.). Finally, we can write some kind of script to install all the necessary helm charts and apply our Kubernetes resources.
Because our software deals with sensitive data and makes our business, we need to be careful when deploying a new release. Therefore, we want to somehow test it before we release it, which is very easy to do on Kubernetes clusters. Indeed, we can imagine we have two environments, one for testing and one for production. The testing (or staging) environment would be synchronized with our software repository's main
branch while the production environment would be the pendant of our repo's production
branch. We develop on the main
branch and, as soon as the Q&A is satisfied with the software pushed there, we push it to production
.
We are now in the complicated situation where we want to develop our software on a development machine, test it somehow on an almost productive environment, and release it to a production environment. That leads us to three different build and deployment procedures. On a development machine, we surely want to interact with a short-lived database. Moreover, login credentials to our microservices (like the assets service) should be trivial. On staging, we might want to grant unprotected access to some of our services, for the sake of debugging. On production, we want to secure and hide as much as possible.
Finally, if our development environment was close to the production environment, we would minimize the amount of surprises following a deployment to staging or production, which would increase our productivity.
Enter devspace
Devspace is a cli tool that allows automation of both the build and the deployment of container images. In addition, that tool might as well replace our makefile or docker-compose configurations and provides us with the ability to do Kubernetes-based development. Because of the latter ability, let's assume we have set up a small cluster on our development machine. In one click, you can have Jelastic set up that development cluster for you
through a very simple interface
Or you can manually set up your own kind, minikube, or docker for desktop cluster.
The easiest way to install devspace (not on your Kubernetes cluster, on a remote machine from which you develop your code!) is to do
npm install -g devspace
Then, depending on our use-case, we might run
devspace init
and follow the instructions. In our particular case, we want to build
- our API
- a bunch of custom micro-services
That we do with the following configuration:
version: v1beta10
vars:
- name: SOME_IMPORTANT_VARIABLE
source: env
default: the-important-value
images:
my-custom-service:
image: my-repo/my-custom-service
tags:
- ${DEVSPACE_RANDOM}
dockerfile: ./my-custom-service/Dockerfile
context: .
build:
docker:
options:
target: app
buildArgs:
SOME_IMPORTANT_VARIABLE: ${SOME_IMPORTANT_VARIABLE}
api:
image: my-repo/api
tags:
- ${DEVSPACE_RANDOM}
dockerfile: ./api/Dockerfile
context: .
The above configuration defines how to build our API and our micro-services. When they are pushed to their docker registry, both docker images will have the same random tag (defined by the built-in variable DEVSPACE_RANDOM
). Instead of using a docker daemon, we can also choose to use custom build commands or kaniko. We can use environment variables, like SOME_IMPORTANT_VARIABLE
and provide the usual options to build docker images.
Next, we want to deploy
- our API
- our custom micro-services
- various third-party services (iam, message queue, faas, assets)
In order to take care of that, we complete the previous configuration with the following snippet:
deployments:
# for the custom service, we have regular k8s manifests
- name: my-custom-service
kubectl:
manifests:
- my-custom-service/manifest.yaml
# for the api, we have written a helm chart
- name: api
helm:
chart:
name: api/chart
values:
image: my-repo/api
postgres:
database: my-database
hostname: postgres
username: my-username
password: my-password
# the database service is a 3rd party
- name: postgres
helm:
chart:
name: postgresql
repo: https://charts.bitnami.com/bitnami
values:
postgresqlDatabase: my-database
postgresqlUsername: my-username
postgresqlPassword: my-password
# the iam service is a 3rd party
- name: iam-service
helm:
chart:
name: fusionauth/fusionauth
values:
database:
protocol: postgresql
user: iam-user
password: iam-password
host: postgres
name: iam-database
user: root-db-username
password: root-db-password
search:
engine: database
The first deployment, my-custom-service
, amounts to
kubectl apply -f my-custom-service/manifest.yaml
The second deployment, api
, is a regular helm installation. Instead of writing our own helm chart, we could have used the built-in component charts which offer a compromise between defining our own helm charts and keeping our Kubernetes resources configuration simple. With our current devspace configuration in place, we can start our development environment:
devspace dev
That command builds our docker images and deploys our software to our development Kubernetes cluster's default
namespace. We are now in a situation where we can develop our code on our development machine and push it to our development Kubernetes cluster. With either hot reloading or auto-reloading, we can even fix our code and the result is automatically propagated to our cluster.
Deploy to multiple environments
Now we have a setup that works for development. We are not very far away from our staging environment setup. First, our docker images need to be tagged following the pattern <my-repo>/<my-service>:staging-<commit-short-sha>
. Second, our staging environment bases on external database and IAM services. Consequently, we don't want to deploy them on staging and we need to adapt the services that depend on them. In devspace, we can define profiles. Until now, our configuration has no reference to any profile, therefore it is the development
profile. We can define the staging
profile, let it base on the development
profile and adapt it as we've just described. To do that, let's add the following configuration to our devspace.yaml
:
profiles:
- name: staging
patches:
# images -> adapt tag
- op: replace
path: /images/0=${DEVSPACE_RANDOM}
value:
- staging-${DEVSPACE_GIT_COMMIT}
# postgres -> remove, we have an external database
- op: remove
path: /deployments/name=postgres
# iam service -> remove, we have an external iam service
- op: remove
path: /deployments/name=iam-service
# api
# -> we need an ingress
- op: replace
path: /deployments/name=api/helm/values/ingress
value:
enabled: true
annotations:
kubernetes.io/ingress.class: nginx-cert
cert-manager.io/cluster-issuer: letsencrypt-prod
hosts:
- host: api-staging.my-staging-domain.com
paths:
- /
tls:
- secretName: api-tls
hosts:
- api-staging.my-staging-domain.com
# -> we need up-to-date database accesses
- op: replace
path: /deployments/name=api/helm/values/postgres
value:
database: my-external-database
hostname: my-external-database-hostname
username: my-external-username
password: my-external-password
# my-custom-service -> nothing to do
We can of course follow the same philosophy coupled with the concept of parent profiles to define our production
profile. Then, building and deploying to staging or production is as simple as
devspace deploy -p staging
devspace deploy -p production
Obviously, remotely debugging those profiles is also possible.
We've only scratched the surface...
Many more features are available, like custom commands definition, port-(reverse-)forwarding, file synchronization, container log streaming, etc., which you can read about here. Wisely used in CI / CD pipelines, devspace can drastically simplify the way you release your software.
Top comments (0)