Inter-frame video compression
The conceptWhy an hour of HD video fits on your phone
Uncompressed, one 1080p frame is 1920 × 1080 pixels × 3 bytes, about 6.2 MB. At 30 frames per second that is about 187 MB every second. Video codecs such as H.264 and AV1 shrink this mostly by not storing each frame from scratch.
Most frames are predicted instead. The codec cuts each frame into blocks, say 16 × 16 pixels, and finds the best-matching patch in an earlier frame. Only now and then is a full picture stored, called a keyframe or I-frame.
For a newsreader turning her head, the block with her ear is almost the same patch from the last frame, shifted 3 pixels left. The codec stores that shift, called a motion vector, plus the small leftover difference, called the residual.
What it’s forScores how well a candidate patch in the previous frame matches the current block, so the encoder can pick the best motion vector.
SAD(dx, dy) = Σ |cur(x, y) − ref(x + dx, y + dy)|
- cur(x, y)
- brightness of a pixel in the block being encoded, e.g. 0–255
- ref
- the earlier frame the decoder already has
- dx, dy
- the candidate shift, the motion vector being tested
- Σ
- add up over every pixel in the block
What it givesA mismatch score; the shift with the lowest score becomes the motion vector.
- Block pixels: 10, 12, 14. Candidate patch: 9, 12, 15.
- Differences: |10−9| + |12−12| + |14−15| = 1 + 0 + 1 = 2.
- A patch scoring 2 beats one scoring 9, so the encoder keeps this shift.