Kubernetes Troubleshooting Guide

Issue 1: CrashLoopBackOff

Symptoms: Pod status shows \CrashLoopBackOff\. Pod keeps restarting.

Diagnosis:

\\\`bash

kubectl describe pod # Check Events section

kubectl logs # Current logs

kubectl logs --previous # Logs from crashed container

\\\`

Root Causes & Fixes:

Issue 2: Pod Stuck in Pending

Symptoms: Pod stays in \Pending\ state, never starts.

\\\`bash

kubectl describe pod # Look at Events

\\\`

Root Causes:

\\\`bash

kubectl describe nodes | grep -A 5 "Allocated resources"

kubectl get pods -A --field-selector=status.phase=Running | awk '{print $1, $2}' | head -20

\\\`

Fix: Add more nodes, reduce resource requests, or delete unused pods

\\\`bash

kubectl get nodes --show-labels

\\\`

\\\`bash

kubectl describe nodes | grep Taints

\\\`

\\\`bash

kubectl get pvc -n

kubectl describe pvc

\\\`

Issue 3: ImagePullBackOff / ErrImagePull

\\\`bash

kubectl describe pod # Shows exact error

\\\`

Root Causes:

\\\`bash

kubectl create secret docker-registry regcred \

--docker-server=registry.io \

--docker-username=user \

--docker-password=pass

# Add to pod spec:

# imagePullSecrets:

# - name: regcred

\\\`

Issue 4: Service Not Routing Traffic

\\\`bash

1. Check service exists and has correct selector

kubectl describe svc myservice

2. Check endpoints (should show pod IPs, not <none>!)

kubectl get endpoints myservice

3. Verify pod labels match service selector

kubectl get pods --show-labels | grep app=myapp

4. Test from inside cluster

kubectl run debug --image=curlimages/curl --rm -it -- \

curl http://myservice.default.svc.cluster.local

\\\`

Most common cause: Labels on pods don't match the Service selector.

Issue 5: Node NotReady

\\\`bash

kubectl describe node # Check conditions and events

SSH to node and check

sudo systemctl status kubelet

sudo journalctl -u kubelet -f

Common fixes

sudo systemctl restart kubelet

sudo systemctl restart containerd

\\\`

Common causes: kubelet crashed, disk pressure (disk > 85%), memory pressure, network plugin issues.

Issue 6: Debugging Toolkit

\\\`bash

Create a debug pod

kubectl run debug --image=ubuntu --rm -it -- bash

Ephemeral container (K8s 1.23+)

kubectl debug -it --image=busybox --target=

Port forward to access service locally

kubectl port-forward svc/myservice 8080:80

Copy files from pod

kubectl cp mypod:/var/log/app.log ./app.log

Check cluster events (sorted by time)

kubectl get events --sort-by='.lastTimestamp' -A

Watch pod status changes

kubectl get pods -w

\\\`