Installing Addons

Add-ons extend your Kubernetes cluster with additional functionality. This guide covers installing and managing add-ons in the Krutrim Kubernetes Service.

What are Add-ons?

Add-ons are additional components that provide essential services or extended functionality to your Kubernetes cluster:

• Core Services: DNS (CoreDNS) for service discovery

• Networking: CNI plugins for pod networking (Cilium, kube-proxy)

• Storage: CSI drivers for persistent storage

• Monitoring: Metrics collection and observability

• Autoscaling: Automatic node and pod scaling

Note: All add-ons, including CoreDNS, must be manually installed after cluster creation.

Critical: CNI Installation

What is CNI?

CNI (Container Network Interface) is required for pod-to-pod communication in your cluster. Without a CNI, your cluster pods cannot communicate.

Kube-proxy vs. Cilium

You have two primary networking options:

Option 1: Cilium (Recommended) ⭐

What is Cilium?

• Modern eBPF-based CNI solution

• High-performance networking

• Built-in network security and observability

• Deployed via Helm chart

Important: Do NOT Install Kube-proxy with Cilium

❌ INCORRECT Configuration:
   Install kube-proxy ➜ Install Cilium ➜ Networking issues

✅ CORRECT Configuration:
   Install Cilium ONLY ➜ Full networking functionality

Why?

• Cilium replaces kube-proxy functionality using eBPF

• Running both can cause conflicts and networking issues

• Cilium provides better performance and features

When to Use Cilium:

• ✅ New clusters

• ✅ Production workloads

• ✅ Need for network security policies

• ✅ High-performance requirements

Option 2: Kube-proxy + Your Own CNI

What is Kube-proxy?

• Traditional Kubernetes networking component

• Provides Service load balancing

• Required if you want to use your own CNI solution

When to Use This Option:

• ✅ Need specific CNI solution (Calico, Flannel, etc.)

• ✅ Have existing expertise with particular CNI

• ✅ Require specific features from other CNI providers

Important: Cilium Add-on May Not Work with Kube-proxy

If you install kube-proxy:
├─ You can install your own CNI solution ✅
├─ DO NOT install Cilium add-on ❌
└─ Cilium is designed to work without kube-proxy

CNI Installation Decision Flow

┌─────────────────────────────────────────┐
│   Do you need a specific CNI solution?  │
└─────────────┬──────────────┬────────────┘
              │              │
        ┌─────▼──────┐  ┌───▼──────┐
        │     NO     │  │   YES    │
        └─────┬──────┘  └───┬──────┘
              │             │
              │             │
    ┌─────────▼────────┐    │
    │  Install Cilium  │    │
    │   (Recommended)  │    │
    └─────────┬────────┘    │
              │             │
    ┌─────────▼────────┐    │
    │  Do NOT install  │    │
    │   kube-proxy     │    │
    └──────────────────┘    │
                            │
              ┌─────────────▼────────────┐
              │  Install kube-proxy first│
              └─────────────┬────────────┘
                            │
              ┌─────────────▼────────────┐
              │ Install your own CNI     │
              │ (Calico, Flannel, etc.)  │
              └──────────────────────────┘

Available Add-ons

The Krutrim Kubernetes Service provides several add-ons to extend cluster functionality.

Core Add-ons

CoreDNS (Recommended)

Description: DNS service for Kubernetes cluster

Features:

• Service discovery and DNS resolution

• Pod-to-service name resolution

• Essential for cluster functionality

Installation:

Add-on: CoreDNS
Version: Default or specific version
Configuration: Default settings

Important: Required for service discovery in your cluster

Verification:

# Check CoreDNS pods are running
kubectl get pods -n kube-system -l k8s-app=kube-dns

# Test DNS resolution
kubectl run -it --rm debug --image=busybox --restart=Never -- nslookup kubernetes.default

Networking Add-ons

Cilium (Recommended)

Description: eBPF-based networking and security solution

Features:

• Pod networking and connectivity

• Service load balancing (replaces kube-proxy)

• Network security policies

• Network observability

• High performance with eBPF

Installation:

Add-on: Cilium
Version: Default or specific version
Configuration: Default settings

Important: DO NOT install kube-proxy when using Cilium

Verification:

# Check Cilium pods are running
kubectl get pods -n kube-system -l k8s-app=cilium

# Verify Cilium status
kubectl exec -n kube-system -ti cilium-xxx -- cilium status

Kube-proxy

Description: Traditional Kubernetes network proxy

Features:

• Service load balancing

• iptables-based networking

• Standard Kubernetes component

When to Install:

• Only if you plan to use your own CNI solution

• Do NOT install if using Cilium add-on

Installation:

Add-on: kube-proxy
Version: Default or specific version
Configuration: Default settings

Important: Install kube-proxy ONLY if using your own CNI solution
Then install your preferred CNI separately using kubectl/helm

Storage Add-ons

CSI

Description: Container Storage Interface driver for persistent storage

Features:

• Persistent Volume (PV) support

• Dynamic volume provisioning

• Storage class management

• Integration with Krutrim Cloud storage

Important: Default Storage Class

Storage class is ONLY available if CSI add-on is enabled.

Without CSI:
├─ No default storage class
├─ Cannot create PersistentVolumeClaims
└─ No dynamic volume provisioning

With CSI:
├─ Default storage class available ✅
├─ Can create PersistentVolumeClaims ✅
└─ Dynamic volume provisioning ✅

Installation:

Add-on: CSI
Version: Default or specific version
Configuration: Default settings

Verification:

# Check storage class is available
kubectl get storageclass

# Expected output:
NAME                 PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE
csi-cinder-sc-qos-delete (default) cinder.csi.openstack.org      Delete          Immediate

# Verify CSI pods
kubectl get pods -n kube-system | grep csi

Usage Example:

# Create a PersistentVolumeClaim
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-app-data
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  # storageClassName: csi-cinder-sc-qos-retain  # Optional, uses default if omitted

Other Add-ons

Cluster Autoscaler

Description: Automatically adjusts node group sizes based on resource demands

Features:

• Automatic node scaling up/down

• Cost optimization

• Resource efficiency

When to Use:

• Variable workload patterns

• Cost-sensitive environments

• Need for automatic capacity management

Metrics Server

Description: Collects resource metrics from kubelets

Features:

• CPU and memory metrics

• Enables kubectl top commands

• Required for Horizontal Pod Autoscaler (HPA)

When to Use:

• Need resource monitoring

• Using Horizontal Pod Autoscaler

• Want to track resource usage with kubectl top

Installing Add-ons: Process Overview

Installation Steps

1

Select Add-on

• Choose the add-on you want to install

• Review add-on description and requirements

2

Configure Add-on

Version Selection:

• Default Version: Recommended for most cases (automatically selected)

• Specific Version: Choose if you need a particular version

• Compatibility: System automatically shows compatible versions for your cluster

Add-on Configuration:

• Most add-ons use sensible defaults

• Advanced configuration typically not needed

• System automatically selects compatible versions

3

Install and Monitor

• Submit the add-on installation

• Typical installation time: 2 minutes

4

Verify Installation

After installation completes:

# List add-ons
kubectl get pods -A

# Check specific add-on (example: Cilium)
kubectl get pods -n kube-system -l k8s-app=cilium

# Verify add-on is functioning
kubectl describe pod <pod-name> -n kube-system

Installation Order Best Practices

1

Create cluster

• Wait for PROVISIONED status

2

Create node groups

• At least 1–2 nodes without taints

• Wait for nodes to be Ready

3

Install CNI (CHOOSE ONE)

• Option A: Cilium ONLY (no kube-proxy) ⭐

• Option B: kube-proxy → Your own CNI

4

Verify cluster functionality

• Check all nodes are Ready

• Check all system pods are Running

5

Install storage (if needed)

• CSI Node Plugin

6

Install optional add-ons

• Cluster Autoscaler

• Metrics Server

• Others as needed

Why This Order Matters:

Cluster → Nodes → CNI → Storage → Others
    ↓        ↓       ↓       ↓        ↓
Required  Required  Required Optional Optional
           for CNI   for     for PVs   for
           to work   system           features

Managing Add-ons

Viewing Installed Add-ons

View list of installed add-ons with:

• Add-on name

• Version

• Status

• Installation date

Common Add-on Scenarios

Scenario 1: New Production Cluster

Recommended Add-ons:

• CoreDNS

• Cilium (without kube-proxy)

• CSI Node Plugin

• Metrics Server

• Cluster Autoscaler (if variable load)

Scenario 2: Development/Testing Cluster

Minimal Add-ons:

• CoreDNS

• Cilium (without kube-proxy)

• CSI Node Plugin (if testing with PVs)

Installation Order:

• CoreDNS → Wait complete

• Cilium → Verify networking

• CSI Node Plugin (optional, only if needed)

Scenario 3: Using Custom CNI

Required Add-ons:

• CoreDNS

• Kube-proxy

• Your CNI (installed separately)

• CSI Node Plugin (if needed)

Installation Order:

• CoreDNS → Wait complete

• Kube-proxy → Wait complete

• Install your CNI using kubectl/helm

• Verify all nodes Ready

• CSI Node Plugin if needed

Troubleshooting Add-on Installation

Add-on Stuck in CREATING

Possible Causes:

• Nodes not ready

• Insufficient resources

• Network connectivity issues

Solution:

# Check node status
kubectl get nodes

# Check add-on pods
kubectl get pods -A

# Describe failing pod
kubectl describe pod <pod-name> -n <namespace>

Add-on Installation Failed

Possible Causes:

• Version incompatibility

• Conflicting add-ons

• Resource constraints

Solution:

1. Check add-on status in UI

2. Review error messages

3. Verify cluster has sufficient resources

4. Remove add-on and try compatible version

Cilium Not Working

Possible Causes:

• Kube-proxy also installed (conflict)

• Nodes not ready

• Insufficient permissions

Solution:

# Check if kube-proxy is running
kubectl get pods -n kube-system -l k8s-app=kube-proxy

# If kube-proxy exists, it conflicts with Cilium

# Remove one or the other

# Check Cilium status
kubectl exec -n kube-system -ti cilium-xxx -- cilium status

# View Cilium logs
kubectl logs -n kube-system -l k8s-app=cilium

Storage Class Not Available

Possible Causes:

• CSI Node Plugin add-on not installed

• CSI pods not running

Solution:

# Check CSI add-on status
kubectl get pods -n kube-system | grep csi

# Install CSI Node Plugin if missing

# Verify storage class
kubectl get storageclass

Add-on Best Practices

✅ Do's

1. Install CoreDNS First: Required for service discovery before other add-ons

2. Install CNI Early: Pod networking is essential for cluster functionality

3. Choose One CNI Strategy: Either Cilium OR kube-proxy + custom CNI

4. Verify After Installation: Check pod status after each add-on

5. Install CSI Node Plugin for Storage: If you need persistent volumes

❌ Don'ts

1. Don't Skip CoreDNS: Service discovery won't work without it

2. Don't Install Cilium with Kube-proxy: They conflict

3. Don't Skip Verification: Always verify add-on is working

4. Don't Install Multiple CNIs: Choose one networking solution

Additional Resources

• Configure Storage - Use your CSI storage class

• Create Load Balancers - Expose services with OCCM

• Best Practices - Cluster optimization guidelines

• Troubleshooting Guide - Common issues and solutions