When a model writes one token at a time for one user, the GPU runs many tiny programs in a row and sits idle waiting between them.

FlyDSL mono kernels, written in Python, fuse a layer's stages into one launch, load the next weights early and pass on partial results as soon as they are ready. AMD's ATOM serving software uses them.

Image models repeat a full pass through the network for every denoising step, even when later steps barely change the picture.

TeaCache tracks how much the image changes between steps and reuses the cached result when the change is small, without retraining. AMD fitted it into an ONNX pipeline on Ryzen AI NPUs.

Every Kubernetes job framework stores its pods, replica counts and status differently, so each scheduler or dashboard needs custom code per framework.

A shared definition format plus a Go library that turns any supported job into the same structural view, used by the KAI Scheduler and Run:ai.

When a GPU program issues many memory loads back to back, the hardware cannot accept them fast enough and instructions wait to start.

Hand-modified assembly experiments that show where these stalls come from and how interleaving loads with maths hides them.