GPUDirect Storage lets GPUs read from storage without a detour through the CPU, but setting it up correctly means working through a lot of documentation.

An open-source Python command-line tool that checks a system, flags problems as FAIL, WARN, INFO or PASS, and suggests fixes. An optional agent skill lets you run it by asking in plain language through a coding assistant.

In mixture-of-experts models, each expert gets a different number of tokens, and that number is only known at run time. Padding every expert to the same size wastes compute and memory.

Two new backends that run all the differently sized matrix multiplications in one operation, called Group-GEMM. One uses AMD's hipBLASLt library; the other generates Triton code that runs on both Nvidia and AMD GPUs.

Much of the time in digital-pathology AI goes into reading huge slide images from disk and cutting them into small tiles, so the GPU sits waiting.

They plugged hipCIM, AMD's GPU-based slide reader, into the MONAI medical AI framework in place of a CPU reader. End-to-end inference became 1.6 to 3.8 times faster on three slides.

Teams want to use whatever GPUs are available, but tooling often assumes one vendor and needs rewriting for another.

A tested combination of Ray 2.58.0, ROCm 10.0.0, PyTorch 2.12 and vLLM on AMD Instinct GPUs, including an example that serves a model behind an OpenAI-compatible endpoint.