github.com
Xinference is a model serving tool that provides a unified production-ready inference API. It allows users to run open-source, speech and multimodal models on cloud infrastructure, on-premises systems or personal laptops by modifying a single line of code.