Open sourcevLLM project
vLLM 0.34
A new release of the popular open-source engine for serving AI models. It adds 10 models, including Mistral Large 4, and sizes its memory cache from what is actually free after warmup.
What’s new: For teams running models on their own GPUs. The old Model Runner V1 is gone and the default block size changed to 64, so check the upgrade notes before updating.
1,061 commits from 412 contributors