Latency Under Load Is a Division, Not a Bandwidth Shortage

Latency under load is not a bandwidth shortage. It is a division, and the divisor is the one number nobody puts on a pricing page: buffer bytes divided by line rate. A 1.5 MB transmit queue on a 200 Mbps port is 60 milliseconds of standing delay. It is 60 milliseconds whether that port is carrying one interactive session or four hundred, and it does not improve when you buy a faster port unless the queue drains faster than the offered load refills it.

This is the bufferbloat mechanism, and separating it from the problem it is usually blamed on matters because the two have different fixes and only one of them costs money. If your bulk transfer is making every interactive request slow, the amount of bandwidth you bought is almost never the variable in control. The queue depth is. Everything below is about that queue: where it lives, how many milliseconds it currently represents on your hardware, what the shipped defaults actually are, and the three fixes worth applying in order.

Throughput and latency are the same pipe viewed twice

A link has exactly two properties that matter. It can move a quantity of data per unit time, and it can deliver a given packet after some delay. When the line is idle, those two properties are independent: your interactive request experiences only propagation delay, because there is nothing ahead of it. When the line is busy, they are no longer independent, because the packet has to wait for everything already in front of it to leave.

That wait is the whole story. A saturated link still carries your interactive packet. It does not drop it, prioritise it, or route around it. It puts it at the back of a line and delivers it late. This is why a speed test reporting 200 Mbps and a voice call stuttering are not contradictory measurements. One is a bulk-throughput number measured on an otherwise idle path. The other is a latency number measured while your own backup job owns the queue.

The two workloads that collide are structurally different, and it helps to name them:

• Bulk — long-lived transfers whose value is total bytes moved. Firmware distribution, image replication, log shipping, media delivery, dataset sync, offsite backup. These care about throughput and are largely indifferent to a few hundred milliseconds of added delay.
• Interactive and control — short request-response exchanges whose value is completion time. API calls, SSH sessions, DNS, TLS handshakes, health checks, database round trips, the small sync traffic inside a multiplayer game, anything a person is waiting on. These move almost no bytes and care about nothing except delay.

The asymmetry is brutal in the bulk workload's favour. A single bulk flow can occupy a queue for as long as it wants, using nearly zero of the interactive workload's byte budget but nearly all of its time budget. A 40 KB API response sitting behind 1.5 MB of file chunks is not bandwidth-starved. It is queued.

The arithmetic you can do in your head

Buffered delay is bytes held divided by drain rate. That is the entire formula, and it is worth internalising because it lets you price a queue before you go looking for one:

milliseconds of standing delay = buffer bytes / (line rate in Mbit/s / 8) / 1000

Run it across the buffer sizes that actually appear on a server and the port speeds that actually appear on a price list, and the shape of the problem becomes obvious:

Two things fall out of that table. The first is that latency per byte improves automatically as ports get faster, which is why upgrading a link often does fix the symptom for a while — and why the symptom usually returns about a month later, once the workload has grown to use the new headroom and the queue is full again. The second is the more useful one: a queue measured in packets represents a time that scales inversely with rate. A 1000-packet ring is 120 milliseconds at 100 Mbps and 12 milliseconds at 1 Gbps, for exactly the same configuration.

That second point is why queue-management schemes that reason in milliseconds are portable and queue limits that reason in packets are not. It is also why the numbers in the next section are worth checking rather than assuming: the same nominal default can be a rounding error on one link and a visible stall on another.

Three queues, not one

When people say "the buffer", they are usually pointing at one device. In practice a packet crosses several queues, and they have different owners, different defaults, and different remedies.

1. The device transmit queue. This is the queue attached to your egress interface, and it is the one you can actually change. On a default Linux build the queueing discipline is pfifo_fast, whose three priority bands are sized to the device's transmit queue length — the value ip link reports as qlen, commonly 1000 on an Ethernet interface. On top of that sit a per-socket cap and, upstream, the first switch hop, which is not yours to tune at all.

2. The per-socket cap. Linux's TCP stack limits how much unacknowledged data a single socket may hand to the queueing layer. The sysctl is tcp_limit_output_bytes and its default is 1,048,576 bytes — 1 MB. This detail is easy to miss and it changes the arithmetic substantially: one bulk flow cannot fill a large queue by itself. At the 1 MB cap, it takes roughly fifteen concurrent bulk flows to occupy a 10240-packet, 14.8 MB default limit. That is the reason a large default limit is less dangerous than it looks, and also the reason a fleet of concurrent transfers is more dangerous than a single big one.

3. The upstream hop. The switch or router your packets reach first has its own buffers, and when your egress rate exceeds that hop's rate, the queue that holds your latency is not on your machine. You cannot configure it. You can only stop offering it more than it can drain, which is a decision about your sender, not about your switch.

There is a fourth constraint worth not confusing with the queue: the receiver's advertised window. How much data can be in flight is bounded by the window, the round-trip time and the sender's pacing, and on long paths that bound can bind before any queue fills. That is a separate failure with a separate diagnostic — the arithmetic behind why a bulk transfer between two continents is window-limited rather than bandwidth-limited — and confusing it with queueing delay sends you tuning the wrong layer.

Why it gets misdiagnosed as a bandwidth problem

Queueing delay produces symptoms that look like every other network problem, and the obvious conclusion is usually the expensive one. The table below is the one worth keeping next to your monitoring:

Two of those rows deserve an extra sentence. The "port upgrade fixed it" row is the one that produces the most confident wrong decisions, because for a few weeks the upgrade genuinely does work. The measurement that would have predicted the relapse is queue occupancy as a fraction of the limit under normal load, and almost nobody watches it. The last row matters for tuning: the constants inside a queue manager are chosen relative to a round-trip time, so a scheme that behaves well on a 10 ms path can behave badly on a 200 ms one and vice versa. That is not a bug in the algorithm, it is the algorithm being told the wrong thing about the world.

The defaults, measured rather than assumed

Here is what ships. The upstream Linux default queueing discipline is pfifo_fast, and the kernel documentation for net.core.default_qdisc describes the knob exactly that way: it exists so you can override "the default of pfifo_fast". The queue depth that discipline allocates is not a fixed number in the source — it is sized from each device's transmit queue length, the qlen 1000 you will see in ip link output on a typical Ethernet interface.

The two disciplines worth deploying override both of those things, and their defaults are documented in the man pages:

Read that table with the previous one in hand. The fq_codel packet limit looks alarming — 14.8 MB of standing delay, 620 milliseconds at 200 Mbps — but it is a hard ceiling reached only under pathological load, not a steady state. Under normal operation occupancy is governed by the 5 ms target, and the 1 MB per-socket cap makes it structurally difficult for a single flow to approach the limit. The honest reading is that the limit is a backstop and the target is the operating point, and it is a mistake to quote either without the other.

One more default is worth flagging because it is not universal. Upstream ships pfifo_fast, but the discipline is a build-time choice, and many distributions and cloud images build with fq_codel or set the sysctl in an init script. Do not assume. tc qdisc show dev <interface> answers the question on the machine in front of you in one line, and it is the single most useful command in this article.

Why the target is milliseconds and not kilobytes

The CoDel family of queue managers controls delay rather than queue length, and that design choice is the most portable thing about it. A 5 ms target means "hold about 5 milliseconds of standing queue, then start signalling". Converting that to bytes depends on the rate, which is exactly the point: at 200 Mbps the default target corresponds to roughly 122 KB of standing queue, and the 100 ms interval to about 2.44 MB. At 10 Gbps the same target is 6.1 MB of bytes and the same latency. The algorithm does not need to be re-tuned when the port speed changes because it never expressed its intent in bytes.

There is a real trade-off baked into those two constants, and the CAKE documentation states it plainly: very short latencies require a very rapid AQM response, but such a rapid response tends to impair throughput when the actual round-trip time is relatively long. Actual RTTs within about an order of magnitude of the configured value work well for both. The same page warns that at the local-area preset the time constants approach the jitter inherent in the kernel's own scheduling, so congestion can be signalled prematurely, flows become sparse and total throughput falls.

Two consequences follow. First, the tunable latency floor a queue manager can deliver is not free; every millisecond below the target is paid for in throughput somewhere. Second, and more important, the queue manager only controls the queueing component of delay. The propagation component is untouchable, and it is set by geography: the arithmetic of why region choice is an arithmetic problem rather than a preferences problem applies here too. Shrinking a queue to nothing does not shorten an ocean.

Three fixes, ordered by what they cost

Apply them in this order. Each one is strictly more expensive than the previous, and the first one resolves a large share of real incidents.

Fix one: put a shallow, fair, AQM-managed queue on your egress device. This costs one configuration line and no money. The intent is to replace a deep FIFO that holds whatever arrives with a queue that keeps delay near the target and gives each flow a fair share:

tc qdisc replace dev eth0 root fq_codel

Confirm it took, and watch the counters:

tc -s qdisc show dev eth0

The output tells you three things worth reading. The configured limits confirm which discipline is actually live. The backlog line shows current queue occupancy — the numerator of the division this article is about. The dropped and overlimits counters tell you whether the AQM is acting at all, and a non-zero requeues count is a hint that the device is contending for the queue. Fair queueing also matters for a reason beyond fairness: separating flows means one bulk transfer cannot dictate the delay experienced by every other flow, which is the property that makes the interactive workload survive.

Fix two: make the bottleneck queue yours by shaping below the line rate. If your sender offers more than the first upstream hop can drain, the queue that holds your latency lives on hardware you do not administer. Shaping your own egress slightly below the real rate moves the bottleneck to a queue you control, where the AQM constants are yours and the delay is bounded:

tc qdisc replace dev eth0 root cake bandwidth 190Mbit

The counter-intuitive part is that deliberately sending slower reduces perceived latency. That is not a paradox; it is the difference between a queue that grows without bound and one that is managed. Set the shaper from the measured sustainable rate rather than the nominal link rate, and set CAKE's round-trip assumption to match your actual users — the presets run from a 100 µs datacentre value to a 300 ms oceanic one, and the right answer is the one that describes your path.

Fix three: reshape the load, or move it. Queueing delay is a scheduling problem before it is a configuration problem. Separating bulk from interactive traffic onto different interfaces, instances or windows is more effective than any queue parameter, because it removes the interaction instead of softening it. The same reasoning governs placement: the interactive half of a mixed workload is the half that cares about distance, which is why the four-line breakdown of a game server's latency budget, in which network ping is usually the smallest line, is a useful template well beyond games. It forces you to attribute delay to the layer that owns it before you spend money on the layer that does not.

Measure it in ten minutes

You do not need a lab. You need one idle measurement and one loaded measurement of the same quantity, taken from the same place.

1. Baseline on an idle link. Record round-trip time to a target near your users with nothing else running. This is the propagation floor plus a small amount of forwarding overhead.
2. Saturate the egress deliberately. Run a real bulk transfer — the actual workload, not a synthetic burst — and let it run long enough that the queue reaches steady occupancy.
3. Measure the same round trip again, while it runs. The difference between the two numbers is the queueing component. If it is tens of milliseconds, the queue is behaving. If it is hundreds or more, you have found the queue that is causing the incident.
4. Read the queue counters. tc -s qdisc show gives backlog and drops; ip -s link gives per-interface transmit counters; ss -tin gives per-connection retransmits, congestion window and pacing rate. A stack trace of where the bytes are is more useful than a single aggregate line.
5. Repeat after each fix. The loaded-minus-idle delta is the metric that responds to a queue change, and it should improve by an order of magnitude when a shallow AQM takes over from a deep FIFO.

The one rule is that both measurements must come from the same vantage point. A ping from your laptop to the server and a ping from the server to somewhere else measure two different paths, and comparing them produces the confident wrong conclusion described earlier.

What no queue configuration can fix

A queue is a local object with local remedies, and it is worth being explicit about the two layers underneath it that no configuration reaches.

The first is propagation. Distance divided by the speed of light in fibre is a floor, and it is not negotiable. This is the reason a well-tuned queue on a badly placed instance will still lose to an untuned queue next to your users, and why placement decisions should be made before tuning decisions rather than after.

The second is the physical network itself, which is less elastic than a cloud console makes it look. At NANOG 98 in October 2026, TeleGeography's Paul Brodsky presented the current state of global infrastructure, and APNIC's write-up of it reports the shape of the market: international bandwidth demand is growing at roughly 30% per year, each annual increment now representing about 2 petabits per second of new in-service capacity, rising to more than 10 petabits per second per year by 2035. New submarine cable investment is running at USD 5 to 6 billion per year across 2026 to 2029. Ownership has concentrated sharply — 38% of cable investment is by four hyperscalers as private systems for their exclusive use, with a further 25% in consortium arrangements they also fund, and the transatlantic route in particular is now about 85% privately funded. Cable systems have a service life of roughly 25 years, and about a quarter of the world's installed subsea cable kilometres are expected to be retired by 2030.

Cloudflare's review of the third quarter of 2026 gives the operational counterpart. Radar recorded 49 disruptions in the quarter: 19 government-directed shutdowns, 14 caused by power failure, and five by natural disasters. Power was the second-largest cause by incident count, and one event is instructive for anyone designing for redundancy — a power failure at a single interconnection facility in Honduras on 10 September affected at least eleven major providers, lasted about three hours, and took national HTTP traffic down 63%. Several of the larger telecom providers in the country were untouched, which Cloudflare attributes to their relying on infrastructure outside it. Spreading across providers buys less than people assume when they all exchange traffic at the same building; that reasoning mirrors the point about why port speed rather than transfer volume is the number that actually limits a deployment. Separately, most disruptions were short: fourteen recovered within an hour and sixteen within three.

The practical reading of both datasets is not pessimism. It is that the queue is the one layer of the stack in this article where a config change produces a measurable, immediate, free improvement, and that the layers below it should be treated as inputs to a placement decision rather than as things to be optimised.

Frequently asked questions

Is this the same as a bandwidth shortfall? No, and the distinction is actionable. A bandwidth shortfall means work is being rejected or delayed because the path cannot carry the total volume. Queueing delay means the work is being carried, but interactive packets are waiting behind bulk packets. The first is solved with capacity or with moving data; the second is solved with queue management and scheduling, at no additional cost.

Does a faster port fix it? It reduces the milliseconds that a given buffer represents, so it often improves the symptom immediately. It does not change the structure. If the offered load grows to refill the new headroom, the delay returns — which is the most common shape of "we already fixed this" incidents. Fix the queue first, then buy capacity if the measurement still calls for it.

Will a shallow queue cost me throughput? It can, and the documentation is explicit that a very rapid congestion response impairs throughput when the real round-trip time is long. That is precisely why the round-trip assumption is configurable. Set it to describe your actual path, measure both throughput and loaded latency afterwards, and treat the pair as the result rather than optimising either alone.

Why does latency get worse when I add more concurrent transfers, even at the same total rate? Because the per-socket cap means one flow cannot fill the queue, so the queue stays partially drained and the AQM's fair queueing keeps flows separated when there are few of them. Many concurrent flows each contribute their share of occupancy, and the aggregate is what the interactive packet waits behind. Concurrency is a multiplier on queue occupancy in a way that raw rate is not.

How do I know whether the queue is mine or my provider's? Compare the queue counters on your own egress device against the loaded latency delta. If your backlog stays flat and your loaded latency still spikes, the queue is upstream of you and the remedy is to stop offering more than the path drains. If your backlog grows during the incident, the queue is on your machine and the fix is local.

Where this becomes an infrastructure decision

Fixes one and two are configuration. Fix three is a procurement question, because reshaping the load only helps if you have somewhere to put the bulk half: a port whose rate you can shape against, at a location close enough that the interactive half's propagation floor is short, with enough NVMe-backed disk that the transfer is not blocked on storage instead of on the network. That combination — a fixed-rate high-bandwidth port, a data centre chosen for where the users actually are, and a flat monthly subscription rather than a metered allowance whose overage would make reshaping the load uneconomic — is the shape of problem the Torrent Simple Application Server line from SurferCloud is built for. It runs 200 Mbps unmetered with no fixed traffic allowance, a dedicated IPv4 address and NVMe system disk at three sizes, in the Los Angeles data centre, on a subscription billed monthly or yearly with no hidden fees — Starter at $4.25/month, Growth at $5.25/month and Pro at $8.05/month, each stated as renewing at the same rate. The full specification and current availability are on the Los Angeles high-bandwidth ULightHost promotion page, and the console remains the authority on price and stock.

The reason this is the right shape rather than the faster port you might reach for first is the arithmetic in the second table. Standing delay is bytes divided by drain rate. A port you can shape below its nominal rate gives you a queue you control; a metered allowance gives you an incentive to saturate it; and a location chosen by price rather than by distance to users leaves you with a propagation floor no queue manager can touch. Get the queue right and a mixed workload becomes what it was supposed to be: a bulk transfer that finishes on schedule and an interactive path that nobody notices is running alongside it.