CPU scheduling

How one CPU core runs hundreds of programs "at once"

It doesn't. A core runs one thread at a time. The operating system's scheduler hands the core to each ready thread in turn for a few milliseconds. The switching is so fast that everything seems to run at the same time.

The scheduler keeps a ready queue of threads that could run now. Threads waiting on disk, network or a keypress sit out until their event arrives. When the core frees up, it picks one and sets its time slice, or quantum.

The policy decides who goes next. First-come-first-served lets one long job block everyone. Shortest-job-first minimises the average wait but must know job lengths. Round robin gives each thread one slice, then sends it to the back.

CPU scheduling

Task Manager's "Set priority" and Linux's nice command

On the Details tab in Windows Task Manager, right-click a process to set its priority, from Low to Realtime. On Linux or macOS, `nice -n 19 make` runs a build at the lowest priority. Both are instructions to the scheduler.

Export a long video at low priority and your browser stays snappy, because the scheduler gives the browser a bigger share of the core. Since Linux 6.6 (2023), the EEVDF scheduler turns nice values into weights that set those shares.

CPU scheduling

A hardware timer stops any program from hogging the core

A program stuck in an infinite loop would never give the core back. So the kernel sets a hardware timer. When it fires, the CPU jumps back into the kernel whatever the program was doing. This is called preemption.

The kernel then does a context switch: it saves the old thread's registers, loads the next thread's saved state and resumes it. Each switch costs time. Short slices feel more responsive, while long slices waste less time switching.

Interactive programs come out ahead on their own. A text editor sleeps until you press a key, so it has used very little CPU. When it wakes, a fair scheduler sees it is owed time and runs it almost at once, ahead of the export.