NvidiaAI inference
Nvidia shows three ways to speed up a large AI model on its DGX Spark desktop
The problem
Running the 124-billion-parameter Ling-3.0-flash model one request at a time was slow because each step spent its time reading model weights from memory.
What they built
They swapped in faster kernels for the expert layers, stored the final output layer in a smaller number format to halve its reads, and let the model draft several tokens at once and check them together.
42.5%faster decoding on GSM8K, with no detectable accuracy change
27 Oct 2026Read the post