Deploying a service mesh in a complex brownfield environment is a lengthy and gradual process requiring upfront planning, or there may exist use cases where you have a specific set of services that either aren't yet ready for migration or for some reason can not be migrated to service mesh.
This blog post will talk about the approaches which can be used to enable services outside of the service mesh to communicate with services within the osm-edge service mesh.
osm-edge forked from Open Service Mesh is a lightweight, extensible, cloud-native, SMI-compatible service mesh built purposely for Edge computing. osm-edge uses lightweight programmable proxy Pipy as a sidecar proxy.
osm-edge offers two ways to allow accessing services within the service mesh:
- via Ingress
- FSM Ingress controller
- Nginx Ingress controller
- Access Control
- Service
- IPRange
The first method to access the services in the service mesh is via Ingress controller, and treat the services outside the mesh as the services inside the cluster. The advantage of this approach is that the setup is simple and straightforward and the disadvantages are also apparent, as you cannot achieve fine-grained access control, and all services outside the mesh can access services within the mesh.
This article will focus on the second approach, which allows support for fine-grained access control on who can access services within the service mesh. This feature is newly added and available in release 1.2.0 osm-edge v1.2.0.
Access Control can be configured via two resource types: Service and IP range. In terms of data transmission, it supports plaintext transmission and mTLS-encrypted traffic.
Let's get started with a demo.
Demo environment preparation
Kubernetes cluster
Using the minimalist Kubernetes distribution k8e:
curl -sfL https://getk8e.com/install.sh | K8E_TOKEN=k8e-mesh INSTALL_K8E_EXEC="server --cluster-init --write-kubeconfig-mode 644 --write-kubeconfig ~/.kube/config" sh -
osm-edge CLI
system=$(uname -s | tr [:upper:] [:lower:])
arch=$(dpkg --print-architecture)
release=v1.2.0
curl -L https://github.com/flomesh-io/osm-edge/releases/download/${release}/osm-edge-${release}-${system}-${arch}.tar.gz | tar -vxzf -
./${system}-${arch}/osm version
cp ./${system}-${arch}/osm /usr/local/bin/
Install osm-edge
Execute the following commands to install the related components of osm-edge.
export osm_namespace=osm-system
export osm_mesh_name=osm
osm install \
--mesh-name "$osm_mesh_name" \
--osm-namespace "$osm_namespace" \
--set=osm.image.pullPolicy=Always
Check to make sure all pods are up and running properly.
Deploy the sample application
#Mock target service
kubectl create namespace httpbin
osm namespace add httpbin
kubectl apply -n httpbin -f https://raw.githubusercontent.com/flomesh-io/osm-edge-docs/main/manifests/samples/httpbin/httpbin.yaml
#Mock external service
kubectl create namespace curl
kubectl apply -n curl -f https://raw.githubusercontent.com/flomesh-io/osm-edge-docs/main/manifests/samples/curl/curl.yaml
#Wait for the dependent POD to start normally
kubectl wait --for=condition=ready pod -n httpbin -l app=httpbin --timeout=180s
kubectl wait --for=condition=ready pod -n curl -l app=curl --timeout=180s
Demo
At this point, we send a request from service curl
to target service httpbin
by executing the following command.
kubectl exec "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}')" -n curl -- curl -sI http://httpbin.httpbin:14001/get
command terminated with exit code 56
The access fails because by default the services outside the mesh cannot access the services inside the mesh and we need to apply an access control policy.
Before applying the policy, you need to enable the access control feature, which is disabled by default.
kubectl patch meshconfig osm-mesh-config -n "$osm_namespace" -p '{"spec":{"featureFlags":{"enableAccessControlPolicy":true}}}' --type= merge
Plaintext transfer
Data can be transferred in plaintext or with two-way TLS encryption. Plaintext transfer is relatively simple, so let's demonstrate the plaintext transfer scenario first.
Service-based access control
First, create a Service
for service curl
.
kubectl apply -n curl -f - <<EOF
apiVersion: v1
kind: Service
metadata:
name: curl
labels:
app: curl
service: curl
spec:
ports:
- name: http
port: 80
selector:
app: curl
EOF
Next, create an access control policy with the source Service
curl
and the target service httpbin
.
kubectl apply -f - <<EOF
kind: AccessControl
apiVersion: policy.openservicemesh.io/v1alpha1
metadata:
name: httpbin
namespace: httpbin
spec:
backends:
- name: httpbin
port:
number: 14001 # targetPort of httpbin service
protocol: http
sources:
- kind: Service
namespace: curl
name: curl
EOF
Execute the command again to send the authentication request, and you can see that this time an HTTP 200
response is received.
kubectl exec "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}')" -n curl -- curl -sI http://httpbin.httpbin:14001/get
HTTP/1.1 200 OK
server: gunicorn/19.9.0
date: Mon, 07 Nov 2022 08:47:55 GMT
content-type: application/json
content-length: 267
access-control-allow-origin: *
access-control-allow-credentials: true
osm-stats-namespace: httpbin
osm-stats-kind: Deployment
osm-stats-name: httpbin
osm-stats-pod: httpbin-69dc7d545c-qphrh
connection: keep-alive
Before continuing with the rest of the demonstration, execute the command kubectl delete accesscontrol httpbin -n httpbin
to delete the policy you just created.
IP range-based access control, plaintext transfer
First, get the pod IP address of the service curl
.
curl_pod_ip="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].status.podIP}')"
Access control using IP ranges is simple, just set the access source type to IPRange
and configure the IP address you just obtained.
kubectl apply -f - <<EOF
kind: AccessControl
apiVersion: policy.openservicemesh.io/v1alpha1
metadata:
name: httpbin
namespace: httpbin
spec:
backends:
- name: httpbin
port:
number: 14001 # targetPort of httpbin service
protocol: http
sources:
- kind: IPRange
name: ${curl_pod_ip}/32
EOF
Execute the command again to test if the control policy is in effect.
kubectl exec "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}')" -n curl -- curl -sI http://httpbin.httpbin:14001/get
HTTP/1.1 200 OK
server: gunicorn/19.9.0
date: Mon, 07 Nov 2022 09:20:57 GMT
content-type: application/json
content-length: 267
access-control-allow-origin: *
access-control-allow-credentials: true
osm-stats-namespace: httpbin
osm-stats-kind: Deployment
osm-stats-name: httpbin
osm-stats-pod: httpbin-69dc7d545c-qphrh
connection: keep-alive
Remember to execute kubectl delete accesscontrol httpbin -n httpbin
to clean up the policy.
The previous ones we used were plaintext transfers, next we look at encrypted transfers.
Encrypted transfers
The default access policy certificate feature is off, turn it on by executing the following command.
kubectl patch meshconfig osm-mesh-config -n "$osm_namespace" -p '{"spec":{"featureFlags":{"enableAccessCertPolicy":true}}}' --type=merge
Create AccessCert
for the access source to assign a certificate for data encryption. The controller will store the certificate information in Secret
curl-mtls-secret
under the namespace curl
, and here also assign SAN curl.curl.cluster.local
for the access source.
kubectl apply -f - <<EOF
kind: AccessCert
apiVersion: policy.openservicemesh.io/v1alpha1
metadata:
name: curl-mtls-cert
namespace: httpbin
spec:
subjectAltNames:
- curl.curl.cluster.local
secret:
name: curl-mtls-secret
namespace: curl
EOF
Redeploy curl
and mount the system-assigned Secret
to the pod.
kubectl apply -n curl -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: curl
spec:
replicas: 1
selector:
matchLabels:
app: curl
template:
metadata:
labels:
app: curl
spec:
serviceAccountName: curl
nodeSelector:
kubernetes.io/os: linux
containers:
- image: curlimages/curl
imagePullPolicy: IfNotPresent
name: curl
command: ["sleep", "365d"]
volumeMounts:
- name: curl-mtls-secret
mountPath: "/certs"
readOnly: true
volumes:
- name: curl-mtls-secret
secret:
secretName: curl-mtls-secret
EOF
Service-based Access Control
The next step is to create an access control policy that uses encrypted transport. When configuring the target service, enable client certificate checking by specifying tls.skipClientCertValidation = false
. For the access source, in addition to the Service
type of access source, the SAN curl.curl.cluster.local
should be specified via AuthenticatedPrincipal
.
kubectl apply -f - <<EOF
kind: AccessControl
apiVersion: policy.openservicemesh.io/v1alpha1
metadata:
name: httpbin
namespace: httpbin
spec:
backends:
- name: httpbin
port:
number: 14001 # targetPort of httpbin service
protocol: http
tls:
skipClientCertValidation: false
sources:
- kind: Service
namespace: curl
name: curl
- kind: AuthenticatedPrincipal
name: curl.curl.cluster.local
EOF
Test whether the access policy is effective. When sending the request, we have to specify the CA certificate, key and certificate to be used for the curl
command of the access source, which will be responded to by HTTP 200
normally.
kubectl exec "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}')" -n curl -- curl -ksI https://httpbin.httpbin:14001/get --cacert /certs/ca.crt --key /certs/tls.key --cert /certs/tls.crt
HTTP/2 200
server: gunicorn/19.9.0
date: Mon, 07 Nov 2022 10:44:05 GMT
content-type: application/json
content-length: 267
access-control-allow-origin: *
access-control-allow-credentials: true
osm-stats-namespace: httpbin
osm-stats-kind: Deployment
osm-stats-name: httpbin
osm-stats-pod: httpbin-69dc7d545c-qphrh
IP Range-Based Access Control
After service-based access control, IP range-based control is simple. You just need to specify the type of access source as IPRange
and specify the IP address of the access source. As the application curl
is redeployed, its IP address needs to be retrieved (perhaps you have discovered the drawbacks of IP range-based access control).
curl_pod_ip="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].status.podIP}')"
kubectl apply -f - <<EOF
kind: AccessControl
apiVersion: policy.openservicemesh.io/v1alpha1
metadata:
name: httpbin
namespace: httpbin
spec:
backends:
- name: httpbin
port:
number: 14001 # targetPort of httpbin service
protocol: http
tls:
skipClientCertValidation: false
sources:
- kind: IPRange
name: ${curl_pod_ip}/32
- kind: AuthenticatedPrincipal
name: curl.curl.cluster.local
EOF
Send the request again for testing.
kubectl exec "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}')" -n curl -- curl -ksI https://httpbin.httpbin:14001/get --cacert /certs/ca.crt --key /certs/tls.key --cert /certs/tls.crt
HTTP/2 200
server: gunicorn/19.9.0
date: Mon, 07 Nov 2022 10:58:55 GMT
content-type: application/json
content-length: 267
access-control-allow-origin: *
access-control-allow-credentials: true
osm-stats-namespace: httpbin
osm-stats-kind: Deployment
osm-stats-name: httpbin
osm-stats-pod: httpbin-69dc7d545c-qphrh
Bingo! The access is successful, indicating that our policy has taken effect
Summary
This blog post will talk about the approaches which can be used to enable services outside of the service mesh to communicate with services within the osm-edge service mesh. There may exist use cases where you have a specific set of services that aren't yet ready for migration to service mesh or for some reason can't be migrated yet.
The access control approaches presented in this article are suitable for solving such problems, and allow you to choose the appropriate type of access source and whether to transfer data in plaintext or encrypted, depending on your needs. The control is more granular than using a unified Ingress for access.
Top comments (0)