Network Configuration

Proper network configuration is crucial for your Kubernetes cluster. This guide explains the key networking concepts and how to configure them correctly.

Network Configuration Components

When creating a cluster, configure these three main network elements:

1

Pod CIDR (Pod IP Address Range)

The Pod CIDR defines the IP address range used by pods running in your cluster.

What is Pod CIDR?

• Definition: The IP address range assigned to pods in your cluster

• Default: 192.168.0.0/16 (if not specified)

• Format: Must be a valid IPv4 CIDR notation

Important Considerations

The range 172.24.0.0/13 is reserved for system use and cannot be used for Pod CIDR.

• Allowed CIDR Ranges:

  • 10.0.0.0/8 - Class A private network

  • 172.16.0.0/12 - Class B private network (excluding reserved 172.24.0.0/13)

  • 192.168.0.0/16 - Class C private network

• No overlap with Node Subnet: Your Pod CIDR must not overlap with the VPC subnet(s) where your nodes are deployed. Nodes use IPs from the VPC subnet, and these ranges must be separate from Pod and Service CIDRs.

• Subnet Allocation: Each node receives a /24 subnet from your Pod CIDR for its pods.

Capacity planning example:

If you configure Pod CIDR as 192.168.0.0/16:
- Total IP addresses available: 65,536
- Each node gets a /24 subnet: 256 IPs per node
- Maximum nodes you can create: 256 nodes
- Each node can run up to 256 pods (minus system overhead)

Planning formula:

Pod CIDR: X.X.X.X/N
Node Subnet: /24 (fixed)
Maximum Nodes = 2^(24-N)

Examples:
- /16 Pod CIDR = 2^(24-16) = 256 nodes
- /17 Pod CIDR = 2^(24-17) = 128 nodes
- /18 Pod CIDR = 2^(24-18) = 64 nodes
- /20 Pod CIDR = 2^(24-20) = 16 nodes

Best Practices for Pod CIDR

• Small Clusters (< 10 nodes):

  • Use /20 or larger: 192.168.0.0/20 — Provides 16 nodes × 256 pods/node

• Medium Clusters (10-50 nodes):

  • Use /18: 192.168.0.0/18 — Provides 64 nodes × 256 pods/node

• Large Clusters (50-256 nodes):

  • Use /16: 192.168.0.0/16 — Provides 256 nodes × 256 pods/node

• Very Large Clusters (> 256 nodes):

  • Use larger ranges like /15 or /14; plan carefully

Important: Once set, the Pod CIDR cannot be changed without recreating the cluster. Plan accordingly!

2

Service CIDR (Service IP Address Range)

The Service CIDR defines the IP address range used by Kubernetes Services.

What is Service CIDR?

• Definition: The IP address range assigned to Kubernetes Services (ClusterIP, NodePort, LoadBalancer)

• Default: 10.100.0.0/16 (if not specified)

• Format: Must be a valid IPv4 CIDR notation

Important Considerations

The range 172.24.0.0/13 is reserved for system use and cannot be used for Service CIDR.

• Allowed CIDR Ranges:

  • 10.0.0.0/8 - Class A private network

  • 172.16.0.0/12 - Class B private network (excluding reserved 172.24.0.0/13)

  • 192.168.0.0/16 - Class C private network

• No overlap with Node Subnet: Your Service CIDR must not overlap with the VPC subnet(s) where your nodes are deployed.

• Usage Pattern:

  • Each Service consumes one IP address

  • IP addresses are assigned sequentially

  • First IP (e.g., 10.100.0.1) is reserved for kubernetes.default service

Capacity planning example:

If you configure Service CIDR as 10.100.0.0/16:
- Total IP addresses available: 65,536
- Maximum Services: ~65,000 (accounting for reserved IPs)

Best Practices for Service CIDR

• Typical Clusters:

  • Use /16: 10.100.0.0/16 — 65,536 service IPs

• Small Deployments:

  • Use /20: 10.100.0.0/20 — 4,096 service IPs

Separation example:

• Good: Pod CIDR: 192.168.0.0/16, Service CIDR: 10.100.0.0/16, Node Subnet: 172.16.1.0/24

• Bad: Overlapping ranges between any of these networks

3

VPC and Subnet Configuration

Your cluster runs within your VPC (Virtual Private Cloud) infrastructure.

VPC KRN (Required)

• What it is: Reference to your VPC where the cluster will be deployed

• Format: krn:vpc:region:account:user:vpc:vpc-id

• Purpose: Defines the network boundary for your cluster

Subnet KRN (Required)

• What it is: Reference to the subnet(s) where cluster resources will be deployed

• Format: krn:vpc:region:account:user:subnet:subnet-id

• Important: The subnet specified here is used when creating LoadBalancer services

LoadBalancer IP allocation:

When you create a Kubernetes Service of type LoadBalancer:
- The LoadBalancer is created in the subnet you specified in Subnet KRN
- One IP address is allocated from that subnet for the LoadBalancer
- This IP is taken from your subnet's available IP pool

Example:

VPC: 10.0.0.0/16
Subnet for Cluster: 10.0.1.0/24 (254 usable IPs)

If you create 10 LoadBalancer services:
- 10 IPs will be used from the 10.0.1.0/24 subnet
- Remaining IPs: 244 available for LoadBalancers or other resources

Best Practices for VPC/Subnet

• Subnet Size: Ensure your subnet has enough IPs for:

  • Node network interfaces

  • LoadBalancer services

  • Other cloud resources Recommended: Use at least a /24 subnet (256 IPs)

• Dedicated Subnets: Consider using dedicated subnets for:

  • Cluster nodes

  • LoadBalancers

  • Application-specific resources

• IP Planning example:

- Subnet for nodes: 10.0.1.0/24 (256 IPs)
- Subnet for LoadBalancers: 10.0.2.0/24 (256 IPs)
- Reserve IPs for future growth

Network Configuration Examples

Example 1: Small Development Cluster

Cluster Name: dev-cluster
Pod CIDR: 192.168.0.0/20 (supports up to 16 nodes)
Service CIDR: 10.100.0.0/20 (4,096 services)
VPC KRN: krn:vpc:region:account:user:vpc:dev-vpc
Subnet KRN: krn:vpc:region:account:user:subnet:dev-subnet-1
Node Subnet (in VPC): 172.16.1.0/24 (for node IPs)

Note: Pod CIDR (192.168.x.x), Service CIDR (10.100.x.x), and Node Subnet (172.16.x.x) 
      are all in different ranges with no overlap.

Use Case:
- 5-10 nodes
- Development and testing
- Limited LoadBalancers

Example 2: Medium Production Cluster

Cluster Name: prod-cluster
Pod CIDR: 192.168.0.0/18 (supports up to 64 nodes)
Service CIDR: 10.100.0.0/16 (65,536 services)
VPC KRN: krn:vpc:region:account:user:vpc:prod-vpc
Subnet KRN: krn:vpc:region:account:user:subnet:prod-subnet-1
Node Subnet (in VPC): 172.16.0.0/24 (for node IPs)

Note: All three ranges are separate and non-overlapping.

Use Case:
- 20-50 nodes
- Production workloads
- Multiple services with LoadBalancers

Example 3: Large Enterprise Cluster

Cluster Name: enterprise-cluster
Pod CIDR: 192.168.0.0/16 (supports up to 256 nodes)
Service CIDR: 10.100.0.0/16 (65,536 services)
VPC KRN: krn:vpc:region:account:user:vpc:enterprise-vpc
Subnet KRN: krn:vpc:region:account:user:subnet:enterprise-subnet-1
Node Subnet (in VPC): 172.16.0.0/23 (for node IPs - larger for more nodes)

Note: Enterprise setup with separate IP ranges for each network layer.

Use Case:
- 100+ nodes
- Large-scale production
- Microservices architecture with many services

Network Configuration Checklist

Before creating your cluster, verify:

• Pod CIDR is sized appropriately for your planned node count

• Service CIDR is large enough for your expected number of services

• No overlap between Pod CIDR and Service CIDR

• No overlap between Pod CIDR and Node Subnet (VPC subnet)

• No overlap between Service CIDR and Node Subnet (VPC subnet)

• Reserved ranges avoided: Not using 172.24.0.0/13

• Using allowed private ranges: 10.0.0.0/8, 172.16.0.0/12 (excluding reserved), or 192.168.0.0/16

• VPC exists and is accessible

• Subnet has sufficient IPs for nodes and LoadBalancers

• Subnet is in the correct VPC

• Network ranges don't conflict with other infrastructure

Common Mistakes to Avoid

❌ Pod CIDR Too Small

Problem: Pod CIDR 192.168.0.0/24 (only 1 node possible)
Solution: Use at least /20 or larger

❌ Using Reserved Range

Problem: Pod CIDR 172.24.0.0/16 (reserved range)
Solution: Use 192.168.0.0/16 or 10.x.x.x ranges

❌ Overlapping CIDRs

Problem 1: Pod and Service CIDR overlap
  Pod CIDR: 10.0.0.0/16
  Service CIDR: 10.0.0.0/16
Solution: Use separate ranges (e.g., Pod: 192.168.0.0/16, Service: 10.100.0.0/16)

Problem 2: Pod CIDR overlaps with Node Subnet
  Pod CIDR: 10.0.0.0/16
  Node Subnet: 10.0.1.0/24 (in VPC)
Solution: Use different ranges (e.g., Pod: 192.168.0.0/16, Node Subnet: 10.0.1.0/24)

Problem 3: Service CIDR overlaps with Node Subnet
  Service CIDR: 172.16.0.0/16
  Node Subnet: 172.16.1.0/24 (in VPC)
Solution: Use different ranges (e.g., Service: 10.100.0.0/16, Node Subnet: 172.16.1.0/24)

❌ Insufficient Subnet IPs

Problem: Small subnet with many LoadBalancers needed
Solution: Use larger subnet (/24 or bigger) or dedicated LoadBalancer subnet

❌ Invalid CIDR Range

Problem: Using invalid or disallowed IP range
  Pod CIDR: 8.8.8.0/24 (public IP range - not allowed)
Solution: Use private IP ranges: 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16

Understanding Network Flow

┌─────────────────────────────────────────────────────────────┐
│                          VPC Network                         │
│                                                              │
│  ┌────────────────────────────────────────────────────────┐ │
│  │            Subnet (Specified in Subnet KRN)            │ │
│  │                                                        │ │
│  │  ┌──────────┐  ┌──────────┐  ┌────────────────────┐ │ │
│  │  │  Node 1  │  │  Node 2  │  │   LoadBalancer     │ │ │
│  │  │ (10.0.1) │  │ (10.0.2) │  │   (10.0.1.100)     │ │ │
│  │  └──────────┘  └──────────┘  └────────────────────┘ │ │
│  │       │              │                  │            │ │
│  └───────┼──────────────┼──────────────────┼────────────┘ │
│          │              │                  │               │
└──────────┼──────────────┼──────────────────┼───────────────┘
           │              │                  │
           │              │                  │
    ┌──────▼──────┐┌──────▼──────┐   ┌──────▼──────┐
    │  Pods using ││  Pods using │   │  External   │
    │  Pod CIDR   ││  Pod CIDR   │   │   Traffic   │
    │ 192.168.0.x ││ 192.168.1.x │   │             │
    └─────────────┘└─────────────┘   └─────────────┘
           │              │
           └──────┬───────┘
                  │
          ┌───────▼────────┐
          │ Services using │
          │  Service CIDR  │
          │  10.100.0.x    │
          └────────────────┘

Next Steps

• Plan your node groups: Managing Nodegroups

• Create your cluster: Creating Cluster

• Configure add-ons: Installing Addons

Need Help?

If you're unsure about network sizing:

• Start with default values for testing

• Monitor your cluster growth

• Contact support for production planning assistance