Installing Addons

Add-ons extend Krutrim Kubernetes Service (KKS) clusters with networking, DNS, persistent storage, resource metrics, and autoscaling capabilities. This guide explains how to select, install, verify, update, and troubleshoot KKS-managed add-ons without exposing underlying infrastructure details.

What are add-ons?

Add-ons are components that run alongside the Kubernetes control plane and workloads. Depending on the component, an add-on can provide:

• Pod networking and Kubernetes Service routing

• Cluster DNS and service discovery

• Dynamic persistent-volume provisioning

• CPU and memory resource metrics

• Automatic node-group scaling

Some add-ons are required for a usable cluster, while others are needed only for specific workloads.

Capability	Typical add-on	When it is needed
Pod networking	Managed CNI	Required before ordinary workload Pods can communicate.
Service discovery	CoreDNS	Required for DNS-based discovery of Kubernetes Services.
Dynamic storage	Managed CSI storage	Required when PVCs must dynamically provision managed persistent storage.
Resource metrics	Metrics Server	Required for kubectl top and resource-metric-based Horizontal Pod Autoscaling.
Node scaling	Cluster Autoscaler	Required when node groups should automatically grow or shrink in response to scheduling demand.

Before you begin

Confirm that:

• The cluster has completed provisioning.

• At least one node group exists.

• Nodes have enough capacity for system add-ons.

• kubectl is configured for the cluster.

• You have permission to manage KKS add-ons.

• You have selected one supported cluster-networking strategy.

Check the cluster and nodes:

kubectl cluster-info
kubectl get nodes -o wide

Nodes may remain NotReady until a CNI is installed. This can be expected during initial networking setup.

Choose a networking strategy

A cluster must use one coherent CNI strategy. Do not install multiple CNIs or combine incompatible Service-routing modes.

flowchart TD
    A[Choose networking] --> B{Use KKS-managed CNI?}
    B -->|Yes| C[Install managed CNI mode]
    B -->|No| D[Validate custom CNI support]
    D --> E[Install required Service proxy and one custom CNI]

KKS-managed CNI mode

Use the managed CNI add-on for the simplest supported configuration. The KKS-managed Cilium option provides Pod networking, network-policy enforcement, and Service routing.

The Service-routing mode is selected by the KKS add-on configuration. If the managed CNI is configured to replace kube-proxy, do not install a separate kube-proxy add-on. If a KKS-supported mode uses kube-proxy, keep that component enabled as directed by the selected configuration.

Do not infer compatibility from generic installation guides. Use only networking combinations exposed and supported by KKS for the cluster version.

Custom CNI mode

Use a custom CNI only when KKS supports the planned integration and your team is prepared to operate it.

Before proceeding, confirm:

• The CNI version supports the cluster's Kubernetes version.

• The required Service-routing mode is enabled.

• Pod and Service CIDRs do not overlap other networks.

• The plugin supports the required network policies, IP families, and observability features.

• The installation procedure does not overwrite KKS-managed resources.

Changing the networking strategy on a running production cluster can disrupt all Pod and Service traffic. Treat a CNI migration as a separate maintenance project with a tested rollback plan.

Available add-ons

The versions shown in the KKS console depend on the Kubernetes version and cluster configuration. Select a compatible version offered by KKS rather than copying a version from another cluster.

CoreDNS

CoreDNS provides DNS-based Kubernetes Service discovery.

Install CoreDNS when workloads need to resolve names such as:

service-name.namespace.svc.cluster.local

CoreDNS Pods require functioning Pod networking. Install and verify the CNI before treating DNS as ready.

Managed CNI

The managed CNI provides:

• Pod-to-Pod connectivity

• Pod-to-Service connectivity

• NetworkPolicy enforcement when configured

• Service routing according to the selected networking mode

• Network observability features supported by the installed version

Install one CNI strategy only.

kube-proxy

kube-proxy implements Kubernetes Service routing on nodes. Install it only when the chosen KKS networking mode or supported custom CNI requires it.

Do not install or remove kube-proxy based solely on the CNI name. Compatibility depends on the selected CNI configuration and Service-routing mode.

Managed CSI storage

The managed CSI storage add-on enables dynamic provisioning of persistent storage through KKS-supported StorageClasses.

With the add-on active:

• KKS can publish a default StorageClass.

• PVCs can dynamically provision managed persistent volumes.

• Expansion and snapshot capabilities depend on the selected StorageClass and cluster version.

Without a dynamic provisioner, Kubernetes can still use administrator-created PVs, but new PVCs are not automatically backed by managed storage.

Metrics Server

Metrics Server provides recent CPU and memory resource metrics through the Kubernetes resource metrics API. It enables:

• kubectl top nodes

• kubectl top pods

• Horizontal Pod Autoscaling based on CPU or memory resource metrics

Metrics Server is not a long-term monitoring, alerting, or historical metrics system. Use a dedicated observability platform for those requirements.

Cluster Autoscaler

Cluster Autoscaler adjusts node-group capacity when Pods cannot be scheduled or when nodes can be removed safely.

Before installing it:

• Configure appropriate minimum and maximum sizes for node groups.

• Ensure workloads have realistic resource requests.

• Review PodDisruptionBudgets and scheduling constraints.

• Confirm that critical system Pods have sufficient capacity.

Cluster Autoscaler changes node capacity. It does not replace Horizontal Pod Autoscaling, which changes workload replica counts.

Recommended installation order

Install and verify one dependency layer before adding the next:

1. Create the cluster and at least one node group.

2. Install the selected CNI and Service-routing mode.

3. Verify node readiness and Pod networking.

4. Install CoreDNS and verify Service discovery.

5. Install managed CSI storage if workloads need dynamic persistent volumes.

6. Install Metrics Server if resource metrics or resource-based HPA are required.

7. Install Cluster Autoscaler if node groups should scale automatically.

8. Install other optional add-ons required by the workload.

flowchart LR
    A[Cluster and nodes] --> B[Networking]
    B --> C[DNS]
    C --> D[Storage]
    D --> E[Metrics and autoscaling]

Installation is asynchronous and can take several minutes. Do not begin a dependent installation until the preceding add-on reports a successful state and its basic function has been verified.

Install an add-on

For each add-on:

1. Open the target KKS cluster and its add-ons section.

2. Select the add-on.

3. Review its purpose, dependencies, networking mode, and supported versions.

4. Choose the default compatible version unless a tested workload requirement calls for another offered version.

5. Review any configuration values before submitting the installation.

6. Start the installation and monitor its status.

7. Wait for the add-on to become active.

8. Run the functional verification for that add-on.

Use default settings when they meet the workload requirements. Change advanced settings only when their effect and upgrade compatibility are understood.

General verification

Check nodes, system Pods, and recent events:

kubectl get nodes -o wide
kubectl get pods --namespace kube-system -o wide
kubectl get events --all-namespaces \
  --sort-by=.metadata.creationTimestamp

Healthy add-on Pods should normally be Running and Ready. Completed setup Jobs can appear as Completed.

For a failing Pod:

kubectl describe pod <POD_NAME> --namespace <NAMESPACE>
kubectl logs <POD_NAME> --namespace <NAMESPACE> --all-containers \
  --tail=200

If a Pod has restarted, inspect the previous container instance:

kubectl logs <POD_NAME> --namespace <NAMESPACE> --all-containers \
  --previous --tail=200

Verify networking

Confirm that nodes are Ready and the managed CNI Pods are healthy:

kubectl get nodes
kubectl get pods --namespace kube-system -o wide

For the KKS-managed Cilium option, you can narrow the Pod list:

kubectl get pods --namespace kube-system -l k8s-app=cilium -o wide

Verify that a CNI configuration exists on every schedulable node and that ordinary Pods can start. If network policies are in use, test both allowed and denied traffic paths before deploying production workloads.

Verify CoreDNS

Check the DNS Pods and Service:

kubectl get pods --namespace kube-system -l k8s-app=kube-dns
kubectl get service --namespace kube-system kube-dns

Run a temporary DNS test Pod:

kubectl run dns-test \
  --image=busybox:1.36.1 \
  --restart=Never \
  --rm -it -- nslookup kubernetes.default.svc.cluster.local

A successful response confirms that the Pod can reach cluster DNS and resolve the default Kubernetes Service.

Verify managed storage

Check the registered CSI drivers and StorageClasses:

kubectl get csidriver
kubectl get storageclass

If a default StorageClass is present, create a small test PVC only when you are prepared for the associated storage allocation and cost:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: storage-test
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

Apply and watch the claim:

kubectl apply -f storage-test-pvc.yaml
kubectl get pvc storage-test --watch

If the StorageClass waits for a consumer, create a test Pod that references the claim before expecting it to bind.

Before deleting the test PVC, inspect its PV and reclaim policy. A Delete policy normally removes the dynamically provisioned storage when the claim is deleted.

Verify Metrics Server

Run:

kubectl top nodes
kubectl top pods --all-namespaces

Metrics may take a short time to appear after installation. If the command reports that the Metrics API is unavailable, inspect the add-on status, Pods, and logs.

Verify Cluster Autoscaler

Confirm that the add-on is active and review autoscaling-related events:

kubectl get events --all-namespaces \
  --sort-by=.metadata.creationTimestamp

No scaling action is expected unless Pods are unschedulable because of node capacity or a node is eligible for safe removal. Verify:

• Node-group minimum and maximum sizes

• Pod resource requests

• Scheduling constraints and taints

• PodDisruptionBudgets

• The reason shown on unschedulable Pods

kubectl describe pod <PENDING_POD_NAME> --namespace <NAMESPACE>

Recommended add-on sets

Production cluster

• One supported CNI and Service-routing mode

• CoreDNS

• Managed CSI storage when persistent storage is required

• Metrics Server when resource metrics or HPA are required

• Cluster Autoscaler when node groups need elastic capacity

• A separate monitoring and alerting platform

Development cluster

• One supported CNI and Service-routing mode

• CoreDNS

• Managed CSI storage only when storage behavior must be tested

• Metrics Server when resource usage or HPA is part of the test

Cluster with a custom CNI

• The required KKS Service-routing mode

• One supported custom CNI

• CoreDNS after networking is operational

• Managed storage and optional add-ons as needed

Validate the custom networking design with Krutrim support before using it in production.

Manage and update add-ons

Review installed add-ons regularly for:

• Installed and available versions

• Compatibility with the cluster's Kubernetes version

• Health and readiness status

• Security and maintenance updates

• Deprecated settings or migration requirements

Before an update:

1. Read the add-on release notes and compatibility information.

2. Confirm that the target version supports the Kubernetes version.

3. Review configuration differences and deprecations.

4. Back up critical application data and configuration.

5. Choose a maintenance window appropriate to the add-on's impact.

6. Update one foundational add-on at a time.

7. Repeat the functional verification after the update.

Do not manually edit KKS-managed add-on Deployments, DaemonSets, Secrets, ConfigMaps, or service accounts. Manual changes can be overwritten during reconciliation or upgrades and can leave the add-on in an unsupported state.

Remove an add-on safely

Add-on removal can interrupt cluster functionality. Review dependencies before uninstalling anything.

• Removing the active CNI can disconnect Pods and make nodes unusable.

• Removing CoreDNS breaks normal Kubernetes Service-name resolution.

• Removing the managed CSI add-on can prevent new provisioning and disrupt future attachment, mounting, expansion, or snapshot operations. Existing data is not a substitute for a tested recovery plan.

• Removing Metrics Server makes kubectl top and resource-metric-based HPA unavailable.

• Removing Cluster Autoscaler stops automatic node-group scaling but does not automatically resize workloads.

Do not remove a foundational add-on from a production cluster unless the replacement has been validated and the migration procedure explicitly supports the transition.

Troubleshooting

Add-on remains in a creating or pending state

Check:

kubectl get nodes
kubectl get pods --all-namespaces -o wide
kubectl get events --all-namespaces \
  --sort-by=.metadata.creationTimestamp

Common causes include:

• No Ready worker nodes

• Insufficient CPU, memory, or Pod capacity

• Untolerated node taints

• Image-pull or registry connectivity failures

• An incompatible add-on version

• A missing prerequisite add-on

• Conflicting networking modes

Add-on installation fails

Review the add-on status and error message, then inspect failed Pods:

kubectl get pods --namespace kube-system
kubectl describe pod <POD_NAME> --namespace kube-system
kubectl logs <POD_NAME> --namespace kube-system \
  --all-containers --tail=200

Do not repeatedly uninstall and reinstall a foundational add-on without understanding the failure. This can compound networking or storage state problems.

Nodes remain NotReady after networking installation

Check node conditions and system Pods:

kubectl describe node <NODE_NAME>
kubectl get pods --namespace kube-system -o wide
kubectl get events --all-namespaces \
  --sort-by=.metadata.creationTimestamp

Confirm that:

• Exactly one CNI strategy is installed.

• The selected Service-routing mode matches the CNI configuration.

• CNI Pods run on every schedulable node.

• Pod and Service CIDRs are correctly configured and non-overlapping.

• Required network traffic between nodes and the control plane is permitted.

DNS resolution fails

Check CoreDNS and the DNS Service:

kubectl get pods --namespace kube-system -l k8s-app=kube-dns
kubectl get service --namespace kube-system kube-dns
kubectl logs --namespace kube-system -l k8s-app=kube-dns --tail=200

Then run the temporary DNS test shown earlier. If the test Pod cannot start or reach the DNS Service, troubleshoot networking before changing DNS configuration.

No default StorageClass appears

Check:

kubectl get csidriver
kubectl get storageclass
kubectl get pods --namespace kube-system | grep -i csi

Confirm that the managed storage add-on is active. A cluster can have registered CSI components without a default StorageClass, so inspect the add-on status and supported configuration rather than inventing a StorageClass.

Resource metrics are unavailable

Check:

kubectl top nodes
kubectl get apiservice | grep metrics
kubectl get pods --namespace kube-system

Metrics Server needs network access to kubelets and valid node information. Use the add-on's supported configuration and collect its Pod events and logs before contacting support.

Cluster Autoscaler does not scale up

Describe a pending Pod and inspect its scheduling events:

kubectl describe pod <PENDING_POD_NAME> --namespace <NAMESPACE>

Scaling might not occur when:

• The node group is already at its maximum size.

• The Pod would not fit on any available node type.

• Node selectors, affinity rules, topology constraints, or taints exclude all node groups.

• Resource requests are missing or unrealistic.

• The pending condition is unrelated to node capacity.

Best practices

• Install one supported CNI strategy and verify it before deploying workloads.

• Install networking before relying on CoreDNS.

• Use only add-on versions offered as compatible with the cluster.

• Install one foundational add-on at a time and verify its function.

• Keep enough node capacity for critical system Pods.

• Use realistic resource requests for add-on and application Pods.

• Treat networking and storage add-on changes as high-impact operations.

• Avoid manual changes to KKS-managed resources.

• Test updates in a non-production cluster first.

• Document the chosen networking mode and add-on versions.

• Monitor add-on Pod readiness, restarts, latency, errors, and capacity.

• Keep a tested backup and recovery plan for stateful workloads.

• Collect status, events, descriptions, and relevant logs before contacting support.

Additional resources

• Kubernetes add-ons

• Kubernetes network plugins

• Debugging DNS resolution

• Resource metrics pipeline

• Kubernetes Persistent Volumes

• Cilium Service-routing modes

• KKS storage configuration

• KKS load balancers