OnCallReady

KubernetessystemdLinux · 1 min read

Node NotReady: kubelet stuck in activating (auto-restart) because swap is on

The kubelet refuses to run with swap enabled by default. How to read it from systemctl and journalctl, why swapoff alone comes back after a reboot, and the full fix.

kubectl get nodes shows a worker NotReady. On the node, the first two commands tell the whole story.

1. systemctl status kubelet

output
● kubelet.service - kubelet: The Kubernetes Node Agent
     Active: activating (auto-restart) (Result: exit-code)

activating (auto-restart) means it started, exited, and systemd is waiting RestartSec before trying again. The kubelet unit is made to retry forever (Restart=always and no start limit), because what makes it fail is usually outside it: the machine isn't set up right yet. So it loops, with no end, instead of landing in failed.

2. journalctl -u kubelet

output
journalctl -u kubelet -n 20 --no-pager

Near the end of each try, the kubelet says why it quit. Here:

output
... failed to run Kubelet: running with swap on is not supported, please disable swap ...

By default the kubelet refuses to start while swap is enabled (failSwapOn: true). Swap makes memory limits and eviction unpredictable. Newer Kubernetes versions can run with swap, but only when you configure that on purpose.

3. The fix has two halves

output
sudo swapoff -a
sudo sed -i '/\sswap\s/ s/^/#/' /etc/fstab
sudo systemctl restart kubelet
  • swapoff -a turns swap off now.
  • Commenting out the swap line in /etc/fstab keeps it off after the next reboot. Skip this and the node goes NotReady again at the next patch reboot.

Check: swapon --show prints nothing, cat /proc/swaps has only its header, systemctl is-active kubelet says active, and the node turns Ready within a minute. Until then, new pods sit in Pending or land on other nodes.

The general lesson

For any unit stuck in activating (auto-restart): systemctl status tells you that it's looping, and journalctl -u <unit> tells you why. The reason is almost always in the last few lines before each restart. On a kubeadm cluster, the other classic NotReady is an expired certificate.

OnCallReady is free, with no ads and no tracking. RSS · All posts