Message queues

How a website can say "uploaded" before your photo is processed

A message queue is a waiting line between two programs. A producer drops in messages (small notes that describe a job), and a consumer takes them out whenever it is ready. Neither one waits for the other, so each runs at its own speed.

Take a photo site. When you upload, the web server queues "make thumbnails for photo 123" and replies in about 50 ms. Workers pull jobs and spend 2 s on each. If 1,000 people upload at once, the jobs just pile up and get worked off.

Message queues

Amazon SQS hides a job while a worker is busy with it

Amazon SQS is AWS's hosted queue service. Producers call SendMessage, and workers call ReceiveMessage to get a job. While a worker holds a message, SQS hides it from other workers for a visibility timeout (30 s by default).

When the worker finishes, it calls DeleteMessage. That call is its acknowledgment, the "done" signal. If the worker crashes first, the timeout runs out, the message reappears, and another worker takes it.

Apache Kafka, first built at LinkedIn, does a similar job. It keeps messages in an ordered log on disk, and many separate consumers can each read that log at their own pace.

Message queues

The queue soaks up rushes so slow workers never lose a job

The producer only waits for one quick write to the queue, so it stays fast however slow the real work is. Messages are kept on disk, often replicated, so jobs survive even if every consumer is down for an hour.

To go faster, run more workers on one queue (competing consumers). Each message goes to just one of them, so 10 workers at 2 s per job give μ = 5 jobs/s. A load balancer pushes requests out; here, free workers pull jobs.

The cost: delivery is at-least-once. A worker that crashes before it acknowledges gets its job rerun, so jobs must be idempotent (safe to run twice). And if λ stays above μ for good, the backlog grows without limit.