Slurm vs Kubernetes: what's the difference?

Slurm and Kubernetes are two pieces of software that manage a cluster of machines. Installed across the whole cluster, each decides which program runs on which machines, and when. Both are free, open-source programs, and neither name is a company. Slurm came out of the United States national laboratories in 2002, Kubernetes out of Google in 2014. A cluster runs one of them, not both: two managers handing out the same machines would each give away what the other had already allocated. The difference between them is the kind of work each was built for. Slurm runs a queue for jobs that finish: ask for machines, wait your turn, run to the end, hand them back. Kubernetes maintains services that must not stop: you state what should always be running, and it starts replacements whenever a machine fails. Slurm still schedules about 60% of the world's top supercomputers; Kubernetes runs much of the software behind the modern web.

Slurm and Kubernetes are both cluster software. A manager program runs on one machine, a small agent runs on every other, and people hand work to the manager rather than choosing machines themselves. The job is the one a laptop's operating system already does: sharing hardware between programs. At the scale of thousands of machines and whole organisations, it adds a question a laptop never faces: whose work has to wait. Slurm, written for the United States national laboratories in 2002, answers with a queue. You ask for machines for a number of hours, wait behind whoever asked earlier, run to the end, and the machines go back to the pool. It was built for supercomputers, where the cluster is a fixed, costly asset that must not sit idle. It still schedules about 60% of the world's top ones. Kubernetes, released by Google in 2014 from its experience running web services, answers with a standing instruction. You declare what should always be true, such as five copies of a service running. Whenever a machine fails, the software makes it true again, starting replacements on the machines left. It was built for work with no end, on machines that come and go. The unit it moves is the container, a program packed with everything it needs so it can start on any machine in seconds. Two answers exist because two industries built them separately. They were only compared once AI needed both: enormous jobs that must finish, and services that must never stop. The real difference is not quality but the question each answers: does the work end, or must it keep going?

Slurm is like the booking sheet for a sports hall; Kubernetes is like the staff roster for a hospital ward. The booking sheet hands out a scarce thing in turns. You get the whole hall from two until four, nobody else can use it while you have it, and at four it goes to the next name on the list. The keeper's whole aim is a full calendar with no empty hours. The ward roster is a different kind of rule: three nurses on duty at all times, day and night, with no finishing date. When one calls in sick, the manager fills the gap at once, because the requirement does not pause while they look. The booking sheet is Slurm, sharing out slots that end. The roster is Kubernetes, keeping a state that must always hold. Both decide who gets a scarce resource.