vLLM

vLLM is an open-source library for efficient serving of large language models. It provides an API server, supports offline and online inference, deployment options, and features such as automatic prefix caching, batch invariance, context extension, and speculative decoding. It can be used with GPUs, CPUs, and TPUs.

Website Directory import
0 shouts Added Sep 23, 2026 docs.vllm.ai
docs.vllm.ai/en/latest/
vLLM website preview

Links

Tags