How to Deploy and Manage Kubernetes Add-Ons across multiple Clusters

Managing multiple Kubernetes clusters across different environments presents significant operational challenges. When you're dealing with numerous EKS clusters running different versions and various Kubernetes add-ons, manual management becomes time-consuming and error-prone. This complexity increases exponentially with each new cluster, leading to configuration drift, inconsistent policies, and deployment headaches.

Understanding Sveltos: The Solution

Sveltos offers a powerful solution to this challenge by providing centralized management of Kubernetes clusters. Instead of installing and managing add-ons on each cluster individually, Sveltos allows you to manage everything from a single control plane. This centralized approach significantly reduces operational overhead and ensures consistency across your entire Kubernetes estate.

Setting Up Your Management Cluster

The first step is establishing your central management cluster. Here's how to get started:

# Create your management cluster (using kind for testing)
kind create cluster --name=control-cluster --config=kind-config.yaml

# Add the Sveltos Helm repository
helm repo add projectsveltos https://projectsveltos.github.io/helm-charts
helm repo update

# Install Sveltos in the management cluster
helm install projectsveltos projectsveltos/projectsveltos -n projectsveltos --create-namespace

Accessing the Sveltos Dashboard

To monitor and manage your clusters, Sveltos provides a dashboard. Set it up with these steps:

# Create a service account for dashboard access
kubectl create sa platform-admin

# Grant necessary permissions
kubectl create clusterrolebinding platform-admin-access \
    --clusterrole cluster-admin \
    --serviceaccount default:platform-admin

# Create an access token
kubectl create token platform-admin --duration=24h

# Make the dashboard accessible
kubectl patch svc dashboard -n projectsveltos -p '{"spec": {"type": "NodePort"}}'
kubectl port-forward svc/dashboard -n projectsveltos 1234:80

Note: The dashboard requires Kubernetes version 1.28.0 or higher for token authentication.

Managing Cluster Add-ons

Sveltos uses ClusterProfiles to manage deployments across clusters. Here's how to deploy add-ons:

apiVersion: config.projectsveltos.io/v1beta1
kind: ClusterProfile
metadata:
  name: deploy-helm-chart
spec:
  clusterSelector:
    matchLabels:
      type: mgmt
  syncMode: Continuous
  helmCharts:
  - repositoryURL: https://helm.nginx.com/stable/
    repositoryName: nginx-stable
    chartName: nginx-stable/nginx-ingress
    chartVersion: 2.0.0
    releaseName: nginx-latest
    releaseNamespace: nginx
    helmChartAction: Install

This configuration allows you to:

• Define which clusters receive the deployment using labels
• Specify Helm charts or raw Kubernetes manifests
• Maintain consistent configurations across clusters
• Automatically sync changes

Best Practices and Tips

When working with Sveltos, keep these points in mind:

1. Label your clusters meaningfully to enable targeted deployments
2. Use the management cluster's dashboard for monitoring deployment status
3. Start with a test environment using kind clusters before moving to production
4. Keep your management cluster on a supported Kubernetes version
5. Organize your ClusterProfiles based on application requirements and environment types

Conclusion

Sveltos transforms the complex task of managing multiple Kubernetes clusters into a streamlined, centralized operation. By providing a single point of control for deploying and managing add-ons across your entire cluster fleet, it significantly reduces operational complexity and ensures consistency in your Kubernetes infrastructure.

For more advanced configurations and detailed documentation, visit the official Sveltos documentation at https://projectsveltos.github.io/sveltos/