OpenLLM

OpenLLM is a project from BentoML for running open-source LLMs or custom models as OpenAI-compatible APIs with a single command. It includes a built-in chat UI, inference backends, and a workflow for cloud deployment with Docker, Kubernetes, and BentoCloud, and it does not store model weights. A Hugging Face token is required for gated models.

GitHub repo Directory import
0 shouts Added Sep 23, 2026 github.com
github.com/bentoml/OpenLLM
OpenLLM website preview

Features

Links

Tags