Scheduling of Pods in Kubernetes

 

🧠 1. Node Selector (Simplest)

This is the most basic way to constrain a Pod to nodes.

👉 You say: “Run this Pod only on nodes with this label.”

spec:
  nodeSelector:
    disktype: ssd

✔ Simple key-value match
❌ No flexibility (no OR, no soft rules)

👉 Think of it as:

“Hard filter: only these nodes allowed”

---

🎯 2. Node Affinity (Advanced Node Selector)

This is a more expressive and powerful version of nodeSelector.

Two types:

✅ Required (Hard rule)

requiredDuringSchedulingIgnoredDuringExecution

Pod must go to matching node or it won’t be scheduled.

🤝 Preferred (Soft rule)

preferredDuringSchedulingIgnoredDuringExecution

Scheduler tries to match but can ignore if needed.

Example:

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: disktype
          operator: In
          values:
          - ssd

✔ Supports operators: In, NotIn, Exists, Gt, etc.
✔ Can express complex logic

👉 Think:

“Smart filtering with flexibility”

---

🚫 3. Pod Affinity / Anti-Affinity

This is about Pod-to-Pod relationships, not nodes.

---

🤝 Pod Affinity (co-location)

👉 “Schedule this Pod near another Pod”

Example:

podAffinity:
  requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchLabels:
          app: redis
      topologyKey: kubernetes.io/hostname

✔ Ensures Pods are on same node / zone

---

❌ Pod Anti-Affinity (separation)

👉 “Do NOT schedule this Pod near similar Pods”

Example:

podAntiAffinity:
  requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchLabels:
          app: web
      topologyKey: kubernetes.io/hostname

✔ Useful for high availability

👉 Think:

• Affinity → “stick together”
• Anti-affinity → “spread apart”

---

⚠️ 4. Taints and Tolerations (Opposite Model)

This is where many people get confused.

👉 Instead of Pods choosing nodes, nodes repel Pods

---

🚫 Taint (on Node)

kubectl taint nodes node1 key=value:NoSchedule

👉 Means:

“Do NOT schedule any Pods here unless they tolerate this”

---

✅ Toleration (on Pod)

tolerations:
- key: "key"
  operator: "Equal"
  value: "value"
  effect: "NoSchedule"

👉 Means:

“This Pod is allowed on tainted nodes”

---

🎯 Effects of Taints

• NoSchedule → don’t place new Pods
• PreferNoSchedule → avoid if possible
• NoExecute → evict existing Pods

---

👉 Think:

• Taint = “Keep out sign 🚫”
• Toleration = “I have permission 🎫”

---

⚙️ 5. Topology Manager

This is more advanced and often overlooked.

👉 It ensures optimal resource alignment on a node, especially for:

• CPU
• NUMA
• GPUs

---

Why needed?

Modern servers have NUMA architecture:

• Memory + CPU split into zones
• Cross-zone access = slower

---

What Topology Manager does:

It coordinates:

• CPU Manager
• Device Manager
• Memory Manager

👉 Goal:

Allocate resources from the same NUMA node

---

Policies:

• none → no alignment
• best-effort → try to align
• restricted → enforce if possible
• single-numa-node → strict alignment

---

👉 Think:

“Even inside a node, placement matters”

---

🔁 How They Fit Together

Feature	Level	Purpose
Node Selector	Node	Simple node filtering
Node Affinity	Node	Advanced node rules
Pod Affinity	Pod-to-Pod	Co-locate Pods
Pod Anti-Affinity	Pod-to-Pod	Spread Pods
Taints	Node	Repel Pods
Tolerations	Pod	Allow exceptions
Topology Manager	Inside Node	Optimize hardware locality

---

🔥 Real-world mental model

Imagine Kubernetes scheduling like this:

1. Node Selector / Affinity → shortlist nodes
2. Taints → remove forbidden nodes
3. Pod Affinity/Anti-Affinity → decide placement relative to other Pods
4. Topology Manager → fine-tune hardware allocation