Latency and throughput
The conceptWhy a fast internet plan can still make web pages feel slow
"Fast" means two different things. Latency is how long one request takes from start to finish, measured in milliseconds. Throughput is how many requests (or bytes) get done per second.
Think of a motorway. Latency is how long your one car takes to drive from A to B. Throughput is how many cars pass a point each hour. Adding lanes raises throughput but doesn't get your car there any sooner.
The two are tied together by Little's law: the work in progress at any moment equals throughput times latency. If you know two of them, you know the third.
What it’s forLinks how much work is in progress to how fast it finishes and how long each piece takes.
L = λ × W
- L
- average number of requests in progress at once (in flight or waiting)
- λ
- throughput: requests completed per second, e.g. 200/s
- W
- latency: average time one request spends in the system, e.g. 0.05 s
What it givesHow many requests a server must hold at once to sustain a given speed.
- λ = 200 requests/s, W = 0.05 s
- L = 200 × 0.05 = 10
- About 10 requests are in progress at any moment