> For the complete documentation index, see [llms.txt](https://docs.cloud.olakrutrim.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.cloud.olakrutrim.com/basics/core-infrastructure/krutrim-kubernetes-system/installing-addons.md). # Installing Addons Add-ons extend Krutrim Kubernetes Service (KKS) clusters with networking, DNS, persistent storage, resource metrics, and autoscaling capabilities. This guide explains how to select, install, verify, update, and troubleshoot KKS-managed add-ons without exposing underlying infrastructure details. ### What are add-ons? Add-ons are components that run alongside the Kubernetes control plane and workloads. Depending on the component, an add-on can provide: * Pod networking and Kubernetes Service routing * Cluster DNS and service discovery * Dynamic persistent-volume provisioning * CPU and memory resource metrics * Automatic node-group scaling Some add-ons are required for a usable cluster, while others are needed only for specific workloads. | Capability | Typical add-on | When it is needed | | ----------------- | ------------------- | ----------------------------------------------------------------------------------------------- | | Pod networking | Managed CNI | Required before ordinary workload Pods can communicate. | | Service discovery | CoreDNS | Required for DNS-based discovery of Kubernetes Services. | | Dynamic storage | Managed CSI storage | Required when PVCs must dynamically provision managed persistent storage. | | Resource metrics | Metrics Server | Required for `kubectl top` and resource-metric-based Horizontal Pod Autoscaling. | | Node scaling | Cluster Autoscaler | Required when node groups should automatically grow or shrink in response to scheduling demand. | ### Before you begin Confirm that: * The cluster has completed provisioning. * At least one node group exists. * Nodes have enough capacity for system add-ons. * `kubectl` is configured for the cluster. * You have permission to manage KKS add-ons. * You have selected one supported cluster-networking strategy. Check the cluster and nodes: ``` kubectl cluster-info kubectl get nodes -o wide ``` Nodes may remain `NotReady` until a CNI is installed. This can be expected during initial networking setup. ### Choose a networking strategy A cluster must use one coherent CNI strategy. Do not install multiple CNIs or combine incompatible Service-routing modes. ``` flowchart TD A[Choose networking] --> B{Use KKS-managed CNI?} B -->|Yes| C[Install managed CNI mode] B -->|No| D[Validate custom CNI support] D --> E[Install required Service proxy and one custom CNI] ``` #### KKS-managed CNI mode Use the managed CNI add-on for the simplest supported configuration. The KKS-managed Cilium option provides Pod networking, network-policy enforcement, and Service routing. The Service-routing mode is selected by the KKS add-on configuration. If the managed CNI is configured to replace `kube-proxy`, do not install a separate `kube-proxy` add-on. If a KKS-supported mode uses `kube-proxy`, keep that component enabled as directed by the selected configuration. Do not infer compatibility from generic installation guides. Use only networking combinations exposed and supported by KKS for the cluster version. #### Custom CNI mode Use a custom CNI only when KKS supports the planned integration and your team is prepared to operate it. Before proceeding, confirm: * The CNI version supports the cluster's Kubernetes version. * The required Service-routing mode is enabled. * Pod and Service CIDRs do not overlap other networks. * The plugin supports the required network policies, IP families, and observability features. * The installation procedure does not overwrite KKS-managed resources. Changing the networking strategy on a running production cluster can disrupt all Pod and Service traffic. Treat a CNI migration as a separate maintenance project with a tested rollback plan. ### Available add-ons The versions shown in the KKS console depend on the Kubernetes version and cluster configuration. Select a compatible version offered by KKS rather than copying a version from another cluster. #### CoreDNS CoreDNS provides DNS-based Kubernetes Service discovery. Install CoreDNS when workloads need to resolve names such as: ``` service-name.namespace.svc.cluster.local ``` CoreDNS Pods require functioning Pod networking. Install and verify the CNI before treating DNS as ready. #### Managed CNI The managed CNI provides: * Pod-to-Pod connectivity * Pod-to-Service connectivity * NetworkPolicy enforcement when configured * Service routing according to the selected networking mode * Network observability features supported by the installed version Install one CNI strategy only. #### kube-proxy `kube-proxy` implements Kubernetes Service routing on nodes. Install it only when the chosen KKS networking mode or supported custom CNI requires it. Do not install or remove `kube-proxy` based solely on the CNI name. Compatibility depends on the selected CNI configuration and Service-routing mode. #### Managed CSI storage The managed CSI storage add-on enables dynamic provisioning of persistent storage through KKS-supported StorageClasses. With the add-on active: * KKS can publish a default StorageClass. * PVCs can dynamically provision managed persistent volumes. * Expansion and snapshot capabilities depend on the selected StorageClass and cluster version. Without a dynamic provisioner, Kubernetes can still use administrator-created PVs, but new PVCs are not automatically backed by managed storage. #### Metrics Server Metrics Server provides recent CPU and memory resource metrics through the Kubernetes resource metrics API. It enables: * `kubectl top nodes` * `kubectl top pods` * Horizontal Pod Autoscaling based on CPU or memory resource metrics Metrics Server is not a long-term monitoring, alerting, or historical metrics system. Use a dedicated observability platform for those requirements. #### Cluster Autoscaler Cluster Autoscaler adjusts node-group capacity when Pods cannot be scheduled or when nodes can be removed safely. Before installing it: * Configure appropriate minimum and maximum sizes for node groups. * Ensure workloads have realistic resource requests. * Review PodDisruptionBudgets and scheduling constraints. * Confirm that critical system Pods have sufficient capacity. Cluster Autoscaler changes node capacity. It does not replace Horizontal Pod Autoscaling, which changes workload replica counts. ### Recommended installation order Install and verify one dependency layer before adding the next: 1. Create the cluster and at least one node group. 2. Install the selected CNI and Service-routing mode. 3. Verify node readiness and Pod networking. 4. Install CoreDNS and verify Service discovery. 5. Install managed CSI storage if workloads need dynamic persistent volumes. 6. Install Metrics Server if resource metrics or resource-based HPA are required. 7. Install Cluster Autoscaler if node groups should scale automatically. 8. Install other optional add-ons required by the workload. ``` flowchart LR A[Cluster and nodes] --> B[Networking] B --> C[DNS] C --> D[Storage] D --> E[Metrics and autoscaling] ``` Installation is asynchronous and can take several minutes. Do not begin a dependent installation until the preceding add-on reports a successful state and its basic function has been verified. ### Install an add-on For each add-on: 1. Open the target KKS cluster and its add-ons section. 2. Select the add-on. 3. Review its purpose, dependencies, networking mode, and supported versions. 4. Choose the default compatible version unless a tested workload requirement calls for another offered version. 5. Review any configuration values before submitting the installation. 6. Start the installation and monitor its status. 7. Wait for the add-on to become active. 8. Run the functional verification for that add-on. Use default settings when they meet the workload requirements. Change advanced settings only when their effect and upgrade compatibility are understood. ### General verification Check nodes, system Pods, and recent events: ``` kubectl get nodes -o wide kubectl get pods --namespace kube-system -o wide kubectl get events --all-namespaces \ --sort-by=.metadata.creationTimestamp ``` Healthy add-on Pods should normally be `Running` and Ready. Completed setup Jobs can appear as `Completed`. For a failing Pod: ``` kubectl describe pod --namespace kubectl logs --namespace --all-containers \ --tail=200 ``` If a Pod has restarted, inspect the previous container instance: ``` kubectl logs --namespace --all-containers \ --previous --tail=200 ``` ### Verify networking Confirm that nodes are Ready and the managed CNI Pods are healthy: ``` kubectl get nodes kubectl get pods --namespace kube-system -o wide ``` For the KKS-managed Cilium option, you can narrow the Pod list: ``` kubectl get pods --namespace kube-system -l k8s-app=cilium -o wide ``` Verify that a CNI configuration exists on every schedulable node and that ordinary Pods can start. If network policies are in use, test both allowed and denied traffic paths before deploying production workloads. ### Verify CoreDNS Check the DNS Pods and Service: ``` kubectl get pods --namespace kube-system -l k8s-app=kube-dns kubectl get service --namespace kube-system kube-dns ``` Run a temporary DNS test Pod: ``` kubectl run dns-test \ --image=busybox:1.36.1 \ --restart=Never \ --rm -it -- nslookup kubernetes.default.svc.cluster.local ``` A successful response confirms that the Pod can reach cluster DNS and resolve the default Kubernetes Service. ### Verify managed storage Check the registered CSI drivers and StorageClasses: ``` kubectl get csidriver kubectl get storageclass ``` If a default StorageClass is present, create a small test PVC only when you are prepared for the associated storage allocation and cost: ``` apiVersion: v1 kind: PersistentVolumeClaim metadata: name: storage-test namespace: default spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi ``` Apply and watch the claim: ``` kubectl apply -f storage-test-pvc.yaml kubectl get pvc storage-test --watch ``` If the StorageClass waits for a consumer, create a test Pod that references the claim before expecting it to bind. Before deleting the test PVC, inspect its PV and reclaim policy. A `Delete` policy normally removes the dynamically provisioned storage when the claim is deleted. ### Verify Metrics Server Run: ``` kubectl top nodes kubectl top pods --all-namespaces ``` Metrics may take a short time to appear after installation. If the command reports that the Metrics API is unavailable, inspect the add-on status, Pods, and logs. ### Verify Cluster Autoscaler Confirm that the add-on is active and review autoscaling-related events: ``` kubectl get events --all-namespaces \ --sort-by=.metadata.creationTimestamp ``` No scaling action is expected unless Pods are unschedulable because of node capacity or a node is eligible for safe removal. Verify: * Node-group minimum and maximum sizes * Pod resource requests * Scheduling constraints and taints * PodDisruptionBudgets * The reason shown on unschedulable Pods ``` kubectl describe pod --namespace ``` ### Recommended add-on sets #### Production cluster * One supported CNI and Service-routing mode * CoreDNS * Managed CSI storage when persistent storage is required * Metrics Server when resource metrics or HPA are required * Cluster Autoscaler when node groups need elastic capacity * A separate monitoring and alerting platform #### Development cluster * One supported CNI and Service-routing mode * CoreDNS * Managed CSI storage only when storage behavior must be tested * Metrics Server when resource usage or HPA is part of the test #### Cluster with a custom CNI * The required KKS Service-routing mode * One supported custom CNI * CoreDNS after networking is operational * Managed storage and optional add-ons as needed Validate the custom networking design with Krutrim support before using it in production. ### Manage and update add-ons Review installed add-ons regularly for: * Installed and available versions * Compatibility with the cluster's Kubernetes version * Health and readiness status * Security and maintenance updates * Deprecated settings or migration requirements Before an update: 1. Read the add-on release notes and compatibility information. 2. Confirm that the target version supports the Kubernetes version. 3. Review configuration differences and deprecations. 4. Back up critical application data and configuration. 5. Choose a maintenance window appropriate to the add-on's impact. 6. Update one foundational add-on at a time. 7. Repeat the functional verification after the update. Do not manually edit KKS-managed add-on Deployments, DaemonSets, Secrets, ConfigMaps, or service accounts. Manual changes can be overwritten during reconciliation or upgrades and can leave the add-on in an unsupported state. ### Remove an add-on safely Add-on removal can interrupt cluster functionality. Review dependencies before uninstalling anything. * Removing the active CNI can disconnect Pods and make nodes unusable. * Removing CoreDNS breaks normal Kubernetes Service-name resolution. * Removing the managed CSI add-on can prevent new provisioning and disrupt future attachment, mounting, expansion, or snapshot operations. Existing data is not a substitute for a tested recovery plan. * Removing Metrics Server makes `kubectl top` and resource-metric-based HPA unavailable. * Removing Cluster Autoscaler stops automatic node-group scaling but does not automatically resize workloads. Do not remove a foundational add-on from a production cluster unless the replacement has been validated and the migration procedure explicitly supports the transition. ### Troubleshooting #### Add-on remains in a creating or pending state Check: ``` kubectl get nodes kubectl get pods --all-namespaces -o wide kubectl get events --all-namespaces \ --sort-by=.metadata.creationTimestamp ``` Common causes include: * No Ready worker nodes * Insufficient CPU, memory, or Pod capacity * Untolerated node taints * Image-pull or registry connectivity failures * An incompatible add-on version * A missing prerequisite add-on * Conflicting networking modes #### Add-on installation fails Review the add-on status and error message, then inspect failed Pods: ``` kubectl get pods --namespace kube-system kubectl describe pod --namespace kube-system kubectl logs --namespace kube-system \ --all-containers --tail=200 ``` Do not repeatedly uninstall and reinstall a foundational add-on without understanding the failure. This can compound networking or storage state problems. #### Nodes remain NotReady after networking installation Check node conditions and system Pods: ``` kubectl describe node kubectl get pods --namespace kube-system -o wide kubectl get events --all-namespaces \ --sort-by=.metadata.creationTimestamp ``` Confirm that: * Exactly one CNI strategy is installed. * The selected Service-routing mode matches the CNI configuration. * CNI Pods run on every schedulable node. * Pod and Service CIDRs are correctly configured and non-overlapping. * Required network traffic between nodes and the control plane is permitted. #### DNS resolution fails Check CoreDNS and the DNS Service: ``` kubectl get pods --namespace kube-system -l k8s-app=kube-dns kubectl get service --namespace kube-system kube-dns kubectl logs --namespace kube-system -l k8s-app=kube-dns --tail=200 ``` Then run the temporary DNS test shown earlier. If the test Pod cannot start or reach the DNS Service, troubleshoot networking before changing DNS configuration. #### No default StorageClass appears Check: ``` kubectl get csidriver kubectl get storageclass kubectl get pods --namespace kube-system | grep -i csi ``` Confirm that the managed storage add-on is active. A cluster can have registered CSI components without a default StorageClass, so inspect the add-on status and supported configuration rather than inventing a StorageClass. #### Resource metrics are unavailable Check: ``` kubectl top nodes kubectl get apiservice | grep metrics kubectl get pods --namespace kube-system ``` Metrics Server needs network access to kubelets and valid node information. Use the add-on's supported configuration and collect its Pod events and logs before contacting support. #### Cluster Autoscaler does not scale up Describe a pending Pod and inspect its scheduling events: ``` kubectl describe pod --namespace ``` Scaling might not occur when: * The node group is already at its maximum size. * The Pod would not fit on any available node type. * Node selectors, affinity rules, topology constraints, or taints exclude all node groups. * Resource requests are missing or unrealistic. * The pending condition is unrelated to node capacity. ### Best practices * Install one supported CNI strategy and verify it before deploying workloads. * Install networking before relying on CoreDNS. * Use only add-on versions offered as compatible with the cluster. * Install one foundational add-on at a time and verify its function. * Keep enough node capacity for critical system Pods. * Use realistic resource requests for add-on and application Pods. * Treat networking and storage add-on changes as high-impact operations. * Avoid manual changes to KKS-managed resources. * Test updates in a non-production cluster first. * Document the chosen networking mode and add-on versions. * Monitor add-on Pod readiness, restarts, latency, errors, and capacity. * Keep a tested backup and recovery plan for stateful workloads. * Collect status, events, descriptions, and relevant logs before contacting support. ### Additional resources * [Kubernetes add-ons](https://kubernetes.io/docs/concepts/cluster-administration/addons/) * [Kubernetes network plugins](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/) * [Debugging DNS resolution](https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/) * [Resource metrics pipeline](https://kubernetes.io/docs/tasks/debug/debug-cluster/resource-metrics-pipeline/) * [Kubernetes Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) * [Cilium Service-routing modes](https://docs.cilium.io/en/stable/network/kubernetes/kubeproxy-free/) * [KKS storage configuration](https://docs.cloud.olakrutrim.com/basics/core-infrastructure/krutrim-kubernetes-system/storage-configuration) * [KKS load balancers](https://docs.cloud.olakrutrim.com/basics/core-infrastructure/krutrim-kubernetes-system/load-balancers) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.cloud.olakrutrim.com/basics/core-infrastructure/krutrim-kubernetes-system/installing-addons.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.