Protecting an Application’s REST Service With Kubernetes RBAC

Ingress With Authentication Support

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-with-auth
annotations:
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: basic-auth

Reverse Proxy With htpasswd

Apiserver Proxy and RBAC

Step-by-Step Walkthrough

minikube start 
kubectl create deployment hello-app --image=gcr.io/google-samples/hello-app:1.0kubectl expose deployment hello-app --type=NodePort --port=8080
kubectl create sa consumerTOKEN=$(kubectl get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='consumer')].data.token}" | base64 --decode)
curl -k -H "Authorization: Bearer $TOKEN" "https://$(minikube ip):8443/api/v1/namespaces/default/services/hello-app/proxy/"
services "hello-app" is forbidden: User "system:serviceaccount:default:consumer" cannot get resource "services/proxy" in API group "" in the namespace "default"
kubectl create role hello-app-proxy --verb get --resource services/proxy --resource-name hello-appkubectl create rolebinding hello-app-consumer --role hello-app-proxy --serviceaccount default:consumer
curl -k -H "Authorization: Bearer $TOKEN" "https://$(minikube ip):8443/api/v1/namespaces/default/services/hello-app/proxy/"
Hello, world!
Version: 1.0.0
Hostname: hello-app-6d7bb985fd-pxv9q

Considerations

  1. The verbs in the role definition make it possible to allow certain HTTP verbs like GET and prohibit others (POST, PUT, PATCH, …).
  2. There is no way to allow access to only specific application endpoints, e.g. /metrics. This can be insufficient when fine-grained access control is needed.
  3. This approach is meant for service to service communication. Most web UIs expect to be served on a host URL directly. Configuring them to work behind a .../api/v1/.../proxy/ URL is either not possible or requires extra work. Additionally, presenting such a URL to the end-user is usually not what you want.
  4. When accessing a headless service with multiple DNS records, the apiserver proxy does load balancing. That means, calling a headless service through the apiserver proxy is significantly different, compared to resolving the headless service DNS to multiple IPs inside the namespace.
  5. Keep in mind that all traffic will go through the apiserver proxy. So if expected traffic is really high, this is potentially not the way to go.

A Real World Example

Conclusions

Appendix: Sub-Resources

SERVER="https://$(minikube ip):8443"TOKEN=$(kubectl get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='default')].data.token}" | base64 --decode)APIS=$(curl -s -k -H "Authorization: Bearer $TOKEN" $SERVER/apis | jq -r '[.groups | .[].name] | join(" ")')# do core resources first, which are at a separate api location
api="core"
curl -s -k -H "Authorization: Bearer $TOKEN" $SERVER/api/v1 | jq -r --arg api "$api" '.resources | .[] | "\($api) \(.name): \(.verbs | join(" "))"'
# now do non-core resources
for api in $APIS; do
version=$(curl -s -k -H "Authorization: Bearer $TOKEN" $SERVER/apis/$api | jq -r '.preferredVersion.version')
curl -s -k -H "Authorization: Bearer $TOKEN" $SERVER/apis/$api/$version | jq -r --arg api "$api" '.resources | .[]? | "\($api) \(.name): \(.verbs | join(" "))"'
done
...
core services/proxy: create delete get patch update
...

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store