Open sourcevLLM project
vLLM 0.34
A new release of the open-source engine for serving AI models. It sizes its memory cache from what is actually free on the GPU, which gives most models a bigger cache.
What’s new: For teams running their own models. Adds 10 models including Mistral Large 4, removes the old Model Runner V1, and changes defaults that can break existing setups.
1,061 commits from 412 contributors