---

Preparing local Cluster with kind

• Install kind.

• Create 3 node kind cluster with default CNI disabled using below YAML.

    kind: Cluster  
    apiVersion: kind.x-k8s.io/v1alpha4  
    nodes:  
      - role: control-plane  
      - role: worker  
      - role: worker  
    networking:  
      disableDefaultCNI: true
  

  kind create cluster --config=kind-config.yaml

• Check kubectl context. kubectl config current-context

• Beacuse their is no CNI in cluster nodes will in NotReady state. kubectl get nodes

---

Cilium Installation

Install the Cilium CLI Tool

• Cilium can be installed in two ways:

  • Cilium CLI tool.

  • Helm chart.

• Install Cilium CLI tool.

• Check CLI and image version. cilium version

Install Cilium in Cluster

• Install Cilium using Cilium CLI. cilium install

• Enable Hubble. cilium hubble enable --ui

• Check status of Cilium components. cilium status

Validate Cilium Operation

• To validate Cilium installation perform connectivty tests. cilium connectivity test --request-timeout 30s --connect-timeout 10s

Examine Cluster with kubectl

• kubectl get nodes

• kubectl get daemonsets --all-namespaces

• kubectl get deployments --all-namespaces

• Cilium deamon set will be running on all three nodes and cilium-operator deployment will be running on a single node.

---

Operational Overview and Components

• When Cilium is installed several operational components are installed.

  

Cilium Operator

• It is reponsible for managing duties that should be performed once for entire cluster.

• Cluster will continue to function even if Operator is temporary unavalible.

Cilium Agent

• Cilium agent runs as a deamonset so that every node has a Cilium agent pod running.

• It mainly performs following fucntions:

  • Intracts with API server to synchronize cluster state.

  • Intracts with Linux kernel by loading eBPF programs and updating eBPF maps.

  • Intracts with Cilium CNI plugin executable via filesystem socket to get notified of newly scheduled workloads.

  • Creates on demand DNS and Envoy proxies as needed based on requested network policy.

  • Creates Hubble gRPC service when Hubble is enabled.

Cilium Client

• Each pod in Cilium agent deamonset comes with CIlium client executable that can be used to inspect the state of Cilium agent and eBPF map resources.

• Client communicates with Cilium agent's REST API from inside of deamonset.

• This Cilium Client executable is not same as Cilium CLI tool.

Cilium CNI Plugin

• CNI plugin is separate from Cilium agent and it is installed as a part of agent deamon set initialization process.

• It is installed on host filesystem and it is used as default CNI.

• CNI pllugin communicates with running agent via filesystem socket.

Hubble Relay

• When Hubble is enabled agents on each node restart to enable Hubble gRPC service to provide node local obervability.

• For cluster wide obervability Hubble relay deployment is added along with two services Hubble Observer service and Hubble Peer service.

• Hubble Relay deployment provides cluster wide observability by acting as an intermediary between the cluster wide Hubble Observer service and the Hubble gRPC services that each Cilium agent provides.

• Hubble peer detects when new Hubble enabled Cilium agents become active in cluster.

Cluster Mesh API server

• The Cluster Mesh API server is an optional deployment that is only installed if Cilium Cluster Mesh feature is enabled.

• Cilium Cluster Mesh allows Kubernetes services to be shared amongst multiple clusters.

• It deploys etcd in each cluster to hold Cilium identites and exposes proxy service for each of etcd stores.

• Cilium agents running in any member of the same Cluster Mesh can use this service to read information about Cilium identity state globally across the mesh.

• This makes it possible to create and access global services that span the Cluster Mesh.

---

Endpoints and Identity

Cilium Endpoints

• All application containers which share a common IP address are grouped together in what Cilium refers to as an Endpoint.

• Endpoints are an internal representation that Cilium uses to efficiently manage container connectivity.

• Kubernetes pods map directly to Cilium Endpoints. As containers in a pod share same network namespace and have common IP.

• Cilium creates an Endpoint for each Kubernetes pod running in the cluster.

Cilium Identity

• All Cilium Endpoints are assigned a label based identity.

• A Cilium Identity is determined by Labels and is unique cluster wise.

• An Endpoint is assigned the identity which matches the Endpoints security relevent Label.

• A numaric identifier assosiated with each identity is used by eBPF programs in fast lookup in network datapath.

• Hubble also uses numaric identifier to provide network observability.

• As network packets enter or leave a node, Cilium’s eBPF programs map the source and destination IP address to the appropriate numeric identifier and then decide which datapath actions should be taken based on policy configuration referencing those numeric identifiers.

• Each Cilium agent is responsible for updating the Identity relevant eBPF maps with numeric identifiers relevant to endpoints running locally on the node by watching for updates to relevant Kubernetes resources.

  

---