$ kubectl get configmaps -n kube-system cilium-config -oyaml | grep bpf-lb-sock-hostns
bpf-lb-sock-hostns-only: "true"
The new experimental ambient mesh
data plane is not supported, as it interferes with the Cilium data plane.
mTLS mode STRICT or PERMISSIVE (default) are not compatible with Cilium HTTP network policy.
To use an HTTP-based network policy (for example, Layer 7 Examples),
you must configure mtls.mode=DISABLE under Istio’s PeerAuthentication.
When using Kubernetes admission webhooks to inject sidecar proxies
together with Cilium overlay mode (VXLAN or GENEVE), istiod pods must be running with hostNetwork: true in order to be reachable
by the API server.
Demo Application
The following guide demonstrates the interaction between Istio’s mTLS mode and
Cilium network policies, including the caveat described in the Istio configuration section.
Prerequisites
Istio is already installed on the local Kubernetes cluster.
Cilium is already installed with the socketLB.hostNamespaceOnly Helm value.
Istio’s istioctl is installed on the local host.
Start by deploying a set of web servers and client applications across three different namespaces:
kubectl create ns red
kubectl -n red apply -f <(curl -s https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes-istio/httpbin.yaml | istioctl kube-inject -f -)
kubectl -n red apply -f <(curl -s https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes-istio/netshoot.yaml | istioctl kube-inject -f -)
kubectl create ns blue
kubectl -n blue apply -f <(curl -s https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes-istio/httpbin.yaml | istioctl kube-inject -f -)
kubectl -n blue apply -f <(curl -s https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes-istio/netshoot.yaml | istioctl kube-inject -f -)
kubectl create ns green
kubectl -n green apply -f https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes-istio/netshoot.yaml
By default, Istio works in PERMISSIVE mode, allowing both Istio-managed and Pods without sidecars
to send and receive traffic between each other. You can test the connectivity between client and server applications
deployed in the preceding example by entering the following commands:
kubectl exec -n red deploy/netshoot -- curl http://httpbin.red/ip -s -o /dev/null -m 1 -w "client 'red' to server 'red': %{http_code}\n"
kubectl exec -n blue deploy/netshoot -- curl http://httpbin.red/ip -s -o /dev/null -m 1 -w "client 'blue' to server 'red': %{http_code}\n"
kubectl exec -n green deploy/netshoot -- curl http://httpbin.red/ip -s -o /dev/null -m 1 -w "client 'green' to server 'red': %{http_code}\n"
kubectl exec -n red deploy/netshoot -- curl http://httpbin.blue/ip -s -o /dev/null -m 1 -w "client 'red' to server 'blue': %{http_code}\n"
kubectl exec -n blue deploy/netshoot -- curl http://httpbin.blue/ip -s -o /dev/null -m 1 -w "client 'blue' to server 'blue': %{http_code}\n"
kubectl exec -n green deploy/netshoot -- curl http://httpbin.blue/ip -s -o /dev/null -m 1 -w "client 'green' to server 'blue': %{http_code}\n"
All commands should complete successfully:
client 'red' to server 'red': 200
client 'blue' to server 'red': 200
client 'green' to server 'red': 200
client 'red' to server 'blue': 200
client 'blue' to server 'blue': 200
client 'green' to server 'blue': 200
You can apply network policies to restrict communication between namespaces.
The following command applies an L4 network policy that restricts communication
in the blue namespace to clients located only in blue and red namespaces.
kubectl -n blue apply -f https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes-istio/l4-policy.yaml
Re-run the same connectivity checks to confirm the expected result:
client 'red' to server 'red': 200
client 'blue' to server 'red': 200
client 'green' to server 'red': 200
client 'red' to server 'blue': 200
client 'blue' to server 'blue': 200
client 'green' to server 'blue': 000
command terminated with exit code 28
You can then decide to enhance the same network policy to perform additional HTTP-based checks.
The following command applies the L7 network policy allowing communication only with the /ip URL path:
kubectl -n blue apply -f https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes-istio/l7-policy.yaml
At this point, all communication with the blue namespace is broken since the Cilium proxy (HTTP) interferes with
Istio’s mTLS-based HTTPs connections:
client 'red' to server 'red': 200
client 'blue' to server 'red': 200
client 'green' to server 'red': 200
client 'red' to server 'blue': 503
client 'blue' to server 'blue': 503
client 'green' to server 'blue': 000
command terminated with exit code 28
To solve the problem, you can disable Istio’s mTLS authentication by configuring a new policy:
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
spec:
mtls:
mode: DISABLE
You must apply this policy to the same namespace where you implement the HTTP-based network policy:
kubectl -n blue apply -f https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes-istio/authn.yaml
Re-run a connectivity check to confirm that communication with the blue namespaces has been restored.
You can verify that Cilium is enforcing the L7 network policy by accessing a different URL path, for example /deny:
$ kubectl exec -n red deploy/netshoot -- curl http://httpbin.blue/deny -s -o /dev/null -m 1 -w "client 'red' to server 'blue': %{http_code}\n"
client 'red' to server 'blue': 403
