Load Average Counts Tasks, Not Time: Reading the Three Numbers Correctly

Load average measures how many tasks are waiting; it does not measure how long they wait. Every alarm built on the three numbers in /proc/loadavg is therefore a queue-depth alarm pretending to be a latency alarm, and the gap between those two things is where most CPU troubleshooting goes wrong.

The number is not useless. It is one of the few kernel figures that has survived four decades, and it answers a real question: how much work is contending for the machine right now. But the question operators actually ask is "why is this slow", and queue depth is only the numerator of that answer. The denominator is throughput, and the number does not contain it.

This piece takes the three numbers apart. It shows what they are computed from, why the same value means different things on different machines, why a high load can coexist with an idle CPU, and which measurable signal actually bounds how long a request waits. By the end you should be able to look at a load of 8 on four cores and say something more precise than "it is busy".

One numerator, three decay constants

The kernel does not compute three separate measurements. It computes one integer, once every five seconds, and feeds that single value into three exponentially weighted averages that decay at different rates.

The integer is the number of tasks in state R (runnable) plus the number of tasks in state D (uninterruptible sleep), summed across every CPU on the machine. That sum is the numerator for all three numbers. The only difference between them is how much of the previous value they retain at each sample.

In the current kernel source, include/linux/sched/loadavg.h defines the three weights as fixed-point fractions with 11 bits of precision: 1884, 2014 and 2037 out of 2048, with a sample every 5 seconds plus one jiffy. Convert those to decay rates and the familiar labels stop being abstract.

Two consequences follow immediately, and both are checkable.

The sample is instantaneous, not averaged. The numerator is a headcount taken at one moment, every five seconds. A process that is runnable for 200 ms out of every five seconds will be caught in the sample only if the sampling instant happens to land inside that window. Short bursts are systematically undercounted, and the amount of undercounting depends on phase alignment, which is why a build server can report a load of 1.5 while four cores are pinned for most of every minute.

The 15-minute figure is a memory, not a measurement. It retains 99.46% of its previous value at each sample. Feed it a step change and it takes a very long time to arrive.

What the three numbers tell you when they disagree

Because all three are fed the same numerator, they cannot disagree about what is happening now. What they disagree about is when it started. That makes the ratio between them a timestamp, not a severity score.

Here is the arithmetic for a concrete case: a machine sitting at a load of 0.5 that steps to 4.0 at time zero and stays there. Applying the retention factors above at each five-second sample gives the following readings.

Read the last row. Fifteen minutes after a sustained quadrupling, the 15-minute average has still only reached two thirds of the new level. An operator watching a dashboard where the 15-minute line is the primary series will conclude the incident is mild, and will keep concluding that for another ten minutes, while the machine is fully saturated the entire time.

The inverse reading is just as useful. When the 15-minute figure is much higher than the 1-minute figure, the machine was under heavier contention recently and has partially recovered. That is not a current problem. It is a residue, and it will keep dragging on capacity graphs for up to half an hour.

One more detail that changes how you should treat the number: because the numerator is a headcount of tasks and not a measure of work, it has no unit. A load of 6 could be six tasks each wanting a full core, or six hundred tasks each wanting one percent of a core. The kernel returns the same integer for both.

Trap one: state D is counted, and state D is not CPU

The numerator includes uninterruptible sleep. The man page for proc_loadavg states it plainly: the figures give the number of jobs "in the run queue (state R) or waiting for disk I/O (state D)".

That inclusion is a design decision, and the kernel source is candid about why it is hard to implement: nr_uninterruptible is not tracked accurately per CPU, because doing so would add a cross-CPU cache-line miss and an atomic operation to the wakeup path. The counter is incremented on whichever CPU the task ran on when it entered the uninterruptible state and decremented on whichever CPU performed the wakeup. Only the sum across all CPUs is correct. In 2025 the type had to be widened back to unsigned long after a change to 32-bit counters allowed it to overflow on machines with heavy task migration, which produced a miscalculated load average.

The practical effect: a filesystem that stops responding produces a load average that climbs steadily while the CPU sits idle. Every process touching the stalled mount enters D and stays there. The scheduler is not the bottleneck. Nothing is being computed at all. The number nevertheless rises, the CPU utilisation graph stays flat, and the two charts contradict each other.

The same mechanism fires in reverse for memory. A process in direct reclaim, waiting for a page to come back from disk, is in D. Under memory pressure with a fast local disk, load rises while both CPU and disk look unremarkable.

So the first question to ask of any high load is not "how high" but "what is in the numerator". If the count of D-state tasks is close to the load figure, the scheduler is innocent and every CPU-focused remediation will waste an afternoon.

Trap two: the same number means different things on different machines

Load is a count of tasks, and tasks compete for a fixed number of CPUs. The comparison only means something after dividing by core count.

Normalise the load to a per-CPU queue depth, and a simple relationship appears. With N runnable tasks on C cores, each task receives C/N of a core while it is running. A task that needs T of CPU time therefore occupies T times N/C of wall clock. The fraction of its wall time spent waiting is 1 minus C/N.

The last two rows are the important ones. A load of 32 on 16 cores and a load of 2 on 16 cores differ by a factor of sixteen in the number and not at all in the experience, because in both cases there is no queue. Any threshold expressed as an absolute load value — alert at 8, page at 16 — is measuring the machine's size as much as its health.

Trap three: queue depth is not wait time

This is the part that makes the number structurally incapable of answering the latency question, and it is ordinary queueing arithmetic rather than a kernel quirk.

Little's law relates the three quantities: the average number in the system equals the arrival rate multiplied by the average time spent in it. Written as a formula, L equals lambda times W. Load gives you L. If you want W — the thing the user is complaining about — you need lambda, and lambda is nowhere in the three numbers.

Two operators looking at the same graph of the same metric can therefore reach opposite conclusions, and both be right, because each is implicitly assuming a different throughput. The number does not contain the information needed to settle it.

There is a second, subtler consequence. Because load counts tasks rather than time, an expensive task and a cheap one contribute identically. Twenty threads doing one millisecond of work each sum to the same load as one thread doing twenty seconds. The first is noise that will be gone before the next sample. The second is a genuine saturation event. The metric returns 20 for both.

The signal that actually bounds latency

If the goal is to know how long work waits, measure how long work waits. The kernel has been recording this the entire time, one field over from the number everyone reads.

With scheduler statistics enabled, each CPU's line in /proc/schedstat carries nine counters. The eighth is the sum of all time tasks spent waiting to run on that processor, in nanoseconds. The ninth is the number of timeslices run on it. Divide one by the other and you get the average time a timeslice waited before it started executing.

awk '/^cpu[0-9]+ /{print $1, "run_delay_ns="$9, "slices="$10}' /proc/schedstat

Sample it twice a minute apart and take the deltas. Field 8 growth divided by field 9 growth is the mean run-queue delay per timeslice for that minute. On a machine with spare capacity it is small, because a task that becomes runnable usually finds a core. As the per-CPU queue depth passes 1.0 it starts to climb, and it keeps climbing roughly in proportion to the depth. That proportionality is the point: this is the quantity that the load average was being used as a proxy for, measured directly.

Per-process figures are in /proc/<pid>/schedstat and have the same first three fields, so a single slow request handler can be interrogated without instrumenting the application: the second field is its lifetime run delay. A service whose p99 latency rose while its CPU usage did not change usually has a growing second field in the handlers that got slower.

For a coarser but cheaper signal, pressure stall information reports the share of wall-clock time during which work could not proceed because of CPU contention. The interface exposes a some line, meaning at least some tasks were stalled, and a full line, meaning all non-idle tasks were stalled simultaneously and cycles were being wasted outright. Both are exported as percentages over ten, sixty and three hundred second windows.

cat /proc/pressure/cpu

One caveat worth knowing before you build a dashboard on it: the documentation notes that CPU full is undefined at the system level and has been reported as zero since kernel 5.13 for backward compatibility. On a bare-metal host, only the some line carries information. Inside a cgroup, both do.

Five things that look like CPU saturation and are not

Almost every misdiagnosis of a high load comes from attributing the numerator to the scheduler when something else put it there. The table below is ordered by how often it turns out to be the answer.

The fourth row deserves emphasis because it inverts the usual direction of the mistake. Contention that never reaches the scheduler is invisible to load: a process blocked on a futex, a connection pool exhausted, a database waiting on a lock. Those tasks sit in interruptible sleep, are not counted, and the load average reports a serene 0.4 while the p99 climbs. This is the single most common way a load-based alarm fails to fire during a real incident.

The second row is the reason build servers and CI runners have famously unreadable load graphs. Compilers fork constantly. A five-second instantaneous sample of a fork-heavy workload is a sample of a Poisson process, and the variance is large enough to make the reading nearly meaningless at low queue depths.

If the machine is a container, the number is not yours

Load average is computed globally and has no per-namespace variant. There is no cgroup-scoped version of /proc/loadavg. A process inside a container that reads the file receives the host's aggregate count, including every task belonging to every other tenant on that kernel.

That produces a specific and recurring false alarm: a container sized at two cores, sitting at eight percent utilisation, reporting a load of 40 because a neighbour is running a batch job. Nothing in the container is contended. Acting on the number means paying for capacity the workload never asked for.

The reverse error is worse. Container-level CPU limits are enforced by the fair-class scheduler's bandwidth control, and a task that has exhausted its quota is throttled — it is not runnable, so it is not in state R, and it is not waiting on I/O, so it is not in state D. Throttled work contributes nothing to the numerator. A container can therefore be badly starved by its own limit while both its utilisation chart and the host load average look calm.

Inside a cgroup, the evidence is local and unambiguous. The cpu.stat file exposes the number of enforcement periods, how many of them ended with throttling, and the total microseconds lost to it. The cpu.pressure file exposes stall percentages scoped to that cgroup's processes only.

cat /sys/fs/cgroup/<path>/cpu.stat

One documented trap applies here: the five bandwidth counters in cpu.stat are non-hierarchical. They account only for throttling caused by that cgroup's own limit, and deliberately exclude throttling inherited from any ancestor cgroup. A workload limited by a parent quota will show a clean cpu.stat at the leaf and still be starved. Read the parent chain when the leaf looks innocent.

A ten-minute diagnostic sequence

The order matters less than doing all of it before touching anything. Each step is cheap and each one eliminates a family of causes.

1. Normalise. Divide all three numbers by core count. Write down three per-CPU depths. Below 1.0 there is no CPU queue, regardless of the absolute figure.
2. Date the change. Compare the 1-minute to the 15-minute. If the 1-minute is far higher, the change is recent and still developing. If the 15-minute dominates, you are looking at a residue.
3. Attribute the numerator. Count D-state processes. If that count is a large fraction of the load, stop the CPU investigation and go look at storage or the network filesystem.
4. Measure the delay. Take two samples of /proc/schedstat a minute apart. Field 8 over field 9 gives milliseconds of wait per timeslice. This is the number that should be on the dashboard.
5. Confirm with pressure. Read cpu.pressure. A low some average with a high load means the queue is short and moving; the load is not costing anyone time.
6. If containerised, scope it. Ignore /proc/loadavg. Read cpu.stat for throttling and cpu.pressure for stalls, then walk up the parent cgroups.
7. Sanity-check with Little's law. Multiply your measured throughput by your measured wait. If the product does not approximate the queue depth you think you have, you have measured two different queues.

Step four is the one that usually ends the discussion. Once run delay per timeslice is on the screen, the question "is this machine saturated" answers itself, and the answer is expressed in the same unit as the user's complaint: milliseconds.

The kernel's own scheduling floor moved this month

One more reason not to infer wait time from queue depth: the relationship between the two is a property of the scheduler, and that property is under active revision.

Linux 7.3 was released on 18 October 2026. In the 7.4 development cycle, scheduler work merged on 22 October specifically targets scheduling latency for short-slice tasks — the workload shape where many brief runnable entities compete, which is precisely the shape where queue depth and delivered latency diverge most. The patch series behind that work was posted in early October by a Linaro engineer and reported promising results for the same scenario.

None of this changes the arithmetic in this article. It changes the constant of proportionality between depth and delay, and it changes it in the direction of lower delay for a given depth. Which is to say: a threshold calibrated against run-queue depth two kernel generations ago is measuring something that has since been redefined. A threshold calibrated against measured run delay is not.

If you maintain load-based alarms, the migration path is not to delete them. It is to keep them as a cheap directional indicator and add one latency-shaped series beside them, so that the day the two disagree you find out from a dashboard rather than from a customer.

Where this lands when you are choosing the machine

All of the above assumes you can measure the host. That assumption is doing a lot of work, and it is the point at which the decision stops being purely diagnostic.

If the CPU count is fixed by a plan you cannot see inside, per-CPU normalisation is guesswork, and the 15-minute average becomes the only reliable series because it is the only one slow enough to survive a shared sampling path. The useful property in that situation is knowing the core count of the plan you are on, having a stated availability commitment to measure against, and being able to change the core count when the measurement says you need to — rather than re-architecting around a host you cannot interrogate.

SurferCloud's Hong Kong node offering is built around that shape. The lightweight tier runs from 1 vCPU and 2 GB up to 4 vCPU and 8 GB with 30 Mbps and an outbound traffic allowance, and the enterprise tier runs from 2 vCPU and 2 GB to 16 vCPU and 32 GB with unmetered traffic and selectable bandwidth. Both are listed with their core counts on the plan card, which is the number you need for step one of the sequence above, and both list renewal at the same price. The page states a 99.95% availability commitment and kernel hot patching applied at the platform level, so a host-layer upgrade does not require a guest reboot.

For a Hong Kong-facing service where the contention question is really "which neighbour am I sharing a kernel with", having the plan's vCPU count on the page and a path upward that does not require a migration is most of what you need to run the diagnostic honestly. The detail is on the Hong Kong node cloud server page, including the current tier list and terms.

Two related pieces from this series are worth reading alongside: the treatment of CPU steal time and what shared versus dedicated allocation really changes, which covers the case where the contention is not yours at all, and the analysis of a full connection tracking table dropping packets while the link sits idle, which is the same failure pattern one layer down: a counter that saturates before the resource it describes appears to.

FAQ

Is load average useless, then? No. It is a good cheap indicator of whether contention is rising, and it is available on every Unix-like system without configuration. It is a bad latency metric and a bad alerting threshold, which is a narrower claim.

Why does my load sometimes exceed my process count? Because it counts runnable kernel scheduling entities, which includes threads. A single multi-threaded process with 40 active worker threads contributes up to 40.

Should I alert on the 1-minute or the 15-minute value? Neither, if the alert is meant to catch latency. If you must use one, use the 1-minute for detection and the 15-minute for trend, and expect the 15-minute to be structurally late by design rather than by misconfiguration.

Why did the load spike and then settle without anything changing? A short-lived process was caught by one sample. With a five-second instantaneous sample, a single fork-heavy burst produces a step that decays with a 41.5-second half-life.

My container reports a load it cannot possibly be causing. Why? The file is not namespaced. You are reading the host's count, which includes every other tenant on that kernel.

What single number should replace it on a dashboard? Run-queue delay per timeslice, from the scheduler statistics counters. It is in the same unit as your latency budget, it needs no assumptions about throughput, and it stays meaningful when the core count changes.

The one-line version

Load average counts tasks, not time, and the three published figures are one count viewed through three decay constants whose half-lives are 41.5 seconds, 3.45 minutes and 10.7 minutes. Normalise by core count, attribute the numerator to states R and D before blaming the CPU, and then measure run delay directly — because that is the only one of these numbers that is denominated in the thing the user is complaining about.

If you want to go further into the same family of problems, the piece on four distinct failures behind one disk-full message applies the same discipline to storage, and the one on why kernel live patching is not a reboot replacement applies it to patching. Each takes a metric that everyone reads and shows which question it cannot answer.