Storage Configuration

This guide covers persistent storage configuration in Krutrim Kubernetes Service using Container Storage Interface (CSI) drivers.

Understanding Storage in Kubernetes

Kubernetes provides several storage abstractions:

Volumes: Temporary storage tied to pod lifecycle
PersistentVolumes (PV): Cluster-level storage resources
PersistentVolumeClaims (PVC): User requests for storage
StorageClass: Dynamic provisioning configuration

CSI Add-on Overview

The Container Storage Interface (CSI) driver enables dynamic storage provisioning in your cluster.

What is CSI?

CSI is a standard interface for exposing block and file storage systems to containerized workloads on Kubernetes.

Benefits:

✅ Dynamic volume provisioning
✅ Automatic volume creation
✅ Volume snapshots
✅ Volume expansion
✅ Integration with cloud storage

Critical: CSI Add-on Requirement

Default storage class is ONLY available if CSI Node Plugin add-on is enabled.

Without CSI:

❌ No default storage class
❌ Cannot create PersistentVolumeClaims
❌ No dynamic volume provisioning

With CSI:

✅ Default storage class available
✅ Can create PersistentVolumeClaims
✅ Dynamic volume provisioning works

Installing CSI Add-on

Install the CSI add-on to enable persistent storage in your cluster.

Configuring Persistent Storage

Prerequisites

Before configuring storage, ensure:

✅ CSI Node Plugin Installed:
- CSI Node Plugin add-on: ACTIVE

Verify Installation

# Check CSI pods are running
kubectl get pods -n kube-system | grep csi

# Expected output:
csi-cinder-nodeplugin-xxxxx     3/3   Running   0   3m
csi-cinder-nodeplugin-yyyyy     3/3   Running   0   3m

# Verify storage class is available
kubectl get storageclass

# Expected output:
NAME                    PROVISIONER                 RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION
csi-cinder-sc-qos-delete (default)   cinder.csi.openstack.org    Delete          Immediate   true

Understanding Storage Classes

Available Storage Classes

Once the CSI add-on is installed, a default storage class is automatically configured for your cluster.

Default Storage Class

After CSI installation, the default storage class is automatically created:

allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
  name: csi-cinder-sc-qos-delete
parameters:
  availability: nova
provisioner: cinder.csi.openstack.org
reclaimPolicy: Delete
volumeBindingMode: Immediate

Storage Class Parameters

Provisioner: cinder.csi.openstack.org

Uses OpenStack Cinder for block storage
Provides persistent volumes backed by cloud storage

Reclaim Policy: Delete

Volumes are deleted when PVC is deleted
Data is permanently removed
Use caution with production data

Volume Binding Mode: Immediate

Volume is created immediately when PVC is created
Does not wait for pod to be scheduled
Volume may be created in different zone than pod

Allow Volume Expansion: true

Volumes can be resized after creation
Requires PVC and pod restart

Using Persistent Storage

Creating a PersistentVolumeClaim

Basic PVC

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-app-data
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  # storageClassName: krutrim-csi  # Optional, uses default if omitted

Apply PVC:

kubectl apply -f pvc.yaml

Check PVC Status:

kubectl get pvc my-app-data

# Expected states:

# Pending → Binding in progress

# Bound → Volume created and bound

Access Modes

ReadWriteOnce (RWO):

Volume can be mounted read-write by a single node
Most common mode
Supported by default storage class

ReadOnlyMany (ROX):

Volume can be mounted read-only by multiple nodes
Less common
Check if supported

ReadWriteMany (RWX):

Volume can be mounted read-write by multiple nodes
Requires special storage types (not block storage)
Not supported by default Cinder CSI

Using PVC in Pods

Pod with Volume Mount

apiVersion: v1
kind: Pod
metadata:
  name: app-with-storage
spec:
  containers:
  - name: app
    image: nginx:latest
    volumeMounts:
    - name: data
      mountPath: /data
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: my-app-data

StatefulSet with Volume Claim Template

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: database
spec:
  serviceName: database
  replicas: 3
  selector:
    matchLabels:
      app: database
  template:
    metadata:
      labels:
        app: database
    spec:
      containers:
      - name: postgres
        image: postgres:14
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 50Gi

Benefits of VolumeClaimTemplates:

Each pod gets its own PVC
PVC name includes pod name (data-database-0, data-database-1, etc.)
Automatic creation and management

Common Storage Scenarios

Scenario 1: Web Application with User Uploads

# PVC for uploads
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: uploads-storage
  namespace: production
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Gi
---
# Deployment using the PVC
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
  namespace: production
spec:
  replicas: 1  # Note: RWO limits to single replica
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: app
        image: my-web-app:latest
        volumeMounts:
        - name: uploads
          mountPath: /app/uploads
      volumes:
      - name: uploads
        persistentVolumeClaim:
          claimName: uploads-storage

Scenario 2: Database with Persistent Storage

# PVC for database
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-data
  namespace: database
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 200Gi
---
# StatefulSet for database
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  namespace: database
spec:
  serviceName: postgres
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:14
        env:
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secret
              key: password
        - name: PGDATA
          value: /var/lib/postgresql/data/pgdata
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
        ports:
        - containerPort: 5432
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: postgres-data

Scenario 3: Shared Configuration Files

# ConfigMap for config files (not PVC)
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  app.conf: |
    server {
      listen 80;
      server_name localhost;
    }
---
# Pod using ConfigMap
apiVersion: v1
kind: Pod
metadata:
  name: app
spec:
  containers:
  - name: app
    image: nginx:latest
    volumeMounts:
    - name: config
      mountPath: /etc/nginx/conf.d
  volumes:
  - name: config
    configMap:
      name: app-config

Note: Use ConfigMap for configuration, PVC for data

# Single PVC shared by containers in same pod
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: shared-data
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: multi-container-app
spec:
  containers:
  - name: writer
    image: writer-app:latest
    volumeMounts:
    - name: shared
      mountPath: /data
  - name: reader
    image: reader-app:latest
    volumeMounts:
    - name: shared
      mountPath: /data
      readOnly: true
  volumes:
  - name: shared
    persistentVolumeClaim:
      claimName: shared-data

Managing Storage

Viewing Storage Resources

# List storage classes
kubectl get storageclass

# List PersistentVolumes
kubectl get pv

# List PersistentVolumeClaims
kubectl get pvc -A

# Describe PVC for details
kubectl describe pvc my-app-data

# Check PVC events
kubectl get events --field-selector involvedObject.name=my-app-data

Expanding Volumes

The default storage class supports volume expansion.

Step: Edit PVC

kubectl edit pvc my-app-data

Step: Increase Size (example)

spec:
  resources:
    requests:
      storage: 20Gi  # Increased from 10Gi

Step: Restart Pod

# Delete pod to trigger remount (for Deployment)
kubectl delete pod <pod-name>

# For StatefulSet
kubectl rollout restart statefulset <statefulset-name>

Verification:

# Check PVC size
kubectl get pvc my-app-data

# Check from inside pod
kubectl exec <pod-name> -- df -h /data

Important notes:

Can only increase size, not decrease
Pod must be restarted for filesystem resize
Some storage backends may take time to expand

Deleting Storage

Delete PVC:

kubectl delete pvc my-app-data

What happens:

PVC is deleted
PV is deleted (due to Delete reclaim policy)
Underlying storage is removed
Data is permanently lost

Warning: Deleting PVC deletes data permanently!

Protecting Important Data:

Option 1: Change Reclaim Policy

# Change PV reclaim policy to Retain
kubectl patch pv <pv-name> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'

Option 2: Backup Before Deletion

# Create snapshot or backup data before deleting

Option 3: Don't Delete PVC
- Keep PVC even if not in use
- PVC doesn't cost money, storage does
- Reattach to new pods when needed

Storage Best Practices

Size Appropriately

Start with reasonable size
Plan for growth (can expand later)
Monitor usage regularly

Use Appropriate Access Modes

ReadWriteOnce for most applications
ReadOnlyMany for shared read-only data
Understand access mode limitations

Plan for Data Persistence

Important data: Use PVCs
Temporary data: Use emptyDir
Configuration: Use ConfigMaps/Secrets

Monitor Storage Usage

# Check PVC usage
kubectl exec <pod-name> -- df -h

# Monitor regularly
# Set up alerts for high usage

Backup Critical Data

Regular backups of databases
Export important data
Test restore procedures

Use StatefulSets for Stateful Apps

Databases, queues, caches
Automatic PVC management
Stable network identities

Anti-Patterns / Don'ts

Don't Use ReadWriteOnce for Multi-Replica Apps

# ❌ Bad: Multiple replicas with RWO
replicas: 3
volumes:
- persistentVolumeClaim:
    claimName: shared-data  # RWO - only one can mount

# ✅ Good: Use StatefulSet with volumeClaimTemplates
# Or use single replica with RWO

Don't Delete PVC Without Backup

Always backup important data first
Verify backups are restorable
Document data recovery procedures

Don't Overprovision Storage

Start reasonable, expand as needed
Storage costs money
Monitor actual usage

Don't Ignore Disk Full Errors

Monitor disk usage
Set up alerts at 80% usage
Expand or clean up before full

Don't Store Secrets in Volumes

Use Kubernetes Secrets
Use proper secret management
Don't write passwords to persistent storage

Troubleshooting Storage

PVC Stuck in Pending

Symptoms:

PVC status remains Pending
No PV created

Possible Causes:

CSI add-on not installed
Storage provisioning in progress
Storage quota exceeded

Solution:

# Check CSI pods
kubectl get pods -n kube-system | grep csi

# Check PVC events
kubectl describe pvc <pvc-name>

# Check if storage provisioning completed
kubectl get pvc <pvc-name>

# Verify storage class exists
kubectl get storageclass

Pod Cannot Mount Volume

Symptoms:

Pod in ContainerCreating state
Events show volume mount errors

Possible Causes:

PVC not bound
Node doesn't have CSI driver
Volume in use by another pod (RWO)

Solution:

# Check PVC status
kubectl get pvc

# Check pod events
kubectl describe pod <pod-name>

# Check CSI node plugin on node
kubectl get pods -n kube-system -l app=csi-cinder-nodeplugin -o wide

# If RWO, ensure only one pod uses volume
kubectl get pods -o wide | grep <pvc-name>

Volume Out of Space

Symptoms:

Application errors writing to disk
Pod logs show "no space left on device"

Solution:

# Check current usage
kubectl exec <pod-name> -- df -h /data

# Expand volume (see Expanding Volumes section)
kubectl edit pvc <pvc-name>

# Increase storage size

# Restart pod

# Or clean up data
kubectl exec <pod-name> -- rm -rf /data/old-files

Storage Class Not Found

Symptoms:

PVC pending with "no storage class found"
kubectl get storageclass shows nothing

Solution:

# Install CSI Node Plugin add-on

# Verify installation
kubectl get storageclass

# Check CSI pods
kubectl get pods -n kube-system | grep csi

Storage Sizing Guidelines

Application Types

Small Applications / Development:

storage: 10-20Gi

Medium Applications / Production:

storage: 50-100Gi

Databases:

# Small database
storage: 100Gi

# Medium database
storage: 200-500Gi

# Large database
storage: 1Ti+

File Storage / Media:

# Image hosting
storage: 200Gi-1Ti

# Video storage
storage: 1Ti+

Monitoring / Logging:

# Prometheus (30 days retention)
storage: 100-200Gi

# Elasticsearch / Logging
storage: 500Gi-2Ti

Additional Resources

Installing Add-ons - Install CSI add-ons
Best Practices - Storage optimization tips
Troubleshooting Guide - Common storage issues

Last updated 2 months ago

Was this helpful?

hashtagUnderstanding Storage in Kubernetes

hashtagCSI Add-on Overview

hashtagWhat is CSI?

hashtagCritical: CSI Add-on Requirement

hashtagInstalling CSI Add-on

hashtagConfiguring Persistent Storage

hashtagPrerequisites

hashtagVerify Installation

hashtagUnderstanding Storage Classes

hashtagAvailable Storage Classes

hashtagDefault Storage Class

hashtagStorage Class Parameters

hashtagUsing Persistent Storage

hashtagCreating a PersistentVolumeClaim

hashtagBasic PVC

hashtagAccess Modes

hashtagUsing PVC in Pods

hashtagPod with Volume Mount

hashtagStatefulSet with Volume Claim Template

hashtagCommon Storage Scenarios

hashtagScenario 1: Web Application with User Uploads

hashtagScenario 2: Database with Persistent Storage

hashtagScenario 3: Shared Configuration Files

hashtagScenario 4: Multiple Containers Sharing Data

hashtagManaging Storage

hashtagViewing Storage Resources

hashtagExpanding Volumes

hashtagDeleting Storage

hashtagStorage Best Practices

hashtagSize Appropriately

hashtagUse Appropriate Access Modes

hashtagPlan for Data Persistence

hashtagMonitor Storage Usage

hashtagBackup Critical Data

hashtagUse StatefulSets for Stateful Apps

hashtagAnti-Patterns / Don'ts

hashtagDon't Use ReadWriteOnce for Multi-Replica Apps

hashtagDon't Delete PVC Without Backup

hashtagDon't Overprovision Storage

hashtagDon't Ignore Disk Full Errors

hashtagDon't Store Secrets in Volumes

hashtagTroubleshooting Storage

hashtagPVC Stuck in Pending

hashtagPod Cannot Mount Volume

hashtagVolume Out of Space

hashtagStorage Class Not Found

hashtagStorage Sizing Guidelines

hashtagApplication Types

hashtagAdditional Resources

Understanding Storage in Kubernetes

CSI Add-on Overview

What is CSI?

Critical: CSI Add-on Requirement

Installing CSI Add-on

Configuring Persistent Storage

Prerequisites

Verify Installation

Understanding Storage Classes

Available Storage Classes

Default Storage Class

Storage Class Parameters

Using Persistent Storage

Creating a PersistentVolumeClaim

Basic PVC

Access Modes

Using PVC in Pods

Pod with Volume Mount

StatefulSet with Volume Claim Template

Common Storage Scenarios

Scenario 1: Web Application with User Uploads

Scenario 2: Database with Persistent Storage

Scenario 3: Shared Configuration Files

Scenario 4: Multiple Containers Sharing Data

Managing Storage

Viewing Storage Resources

Expanding Volumes

Deleting Storage

Storage Best Practices

Size Appropriately

Use Appropriate Access Modes

Plan for Data Persistence

Monitor Storage Usage

Backup Critical Data

Use StatefulSets for Stateful Apps

Anti-Patterns / Don'ts

Don't Use ReadWriteOnce for Multi-Replica Apps

Don't Delete PVC Without Backup

Don't Overprovision Storage

Don't Ignore Disk Full Errors

Don't Store Secrets in Volumes

Troubleshooting Storage

PVC Stuck in Pending

Pod Cannot Mount Volume

Volume Out of Space

Storage Class Not Found

Storage Sizing Guidelines

Application Types

Additional Resources