Search results

OpenAI-compatible LLM inference server for Apple Silicon using MLX. 2-4x faster than Ollama with tool calling and prompt caching.

ray-llm

github.com

Open-source project that provides machine-learning models, research, training resources, or evaluation tools.

Examples showing how to use the OpenAI vision API to run inference on images, video files and webcam streams.

Rockpool

github.com

Open-source project that provides machine-learning models, research, training resources, or evaluation tools.

Provides containers to encapsulate and deploy EdgeML pipelines and applications.

RunThisLLM

runthisllm.com

RunThisLLM provides machine-learning models, research, training resources, or evaluation tools.

MLOps framework to package, deploy, monitor and manage thousands of production machine learning models.

Semantic Cache Router

github.com

Open-source project that provides machine-learning models, research, training resources, or evaluation tools.

SteaMidra - Advanced Steam game setup and management tool featuring manifest handling, Lua integrations, LumaCore deployment, multiplayer fixes, DLC unlocking, backups, game fixes, and an easy-to-use GUI. Educational purposes only.

Python-free Rust inference server with OpenAI API compatibility and hot model swapping.

Sinabs

github.com

Open-source project that provides machine-learning models, research, training resources, or evaluation tools.

SinglebaseCloud supports deploying, serving, monitoring, or operating AI and machine-learning systems.

SocialBu supports deploying, serving, monitoring, or operating AI and machine-learning systems.

Stan supports deploying, serving, monitoring, or operating AI and machine-learning systems.

An MLOps/LLMOps platform for model building, evaluation, and fine-tuning.

Lets you create apps for your ML projects with deceptively simple Python scripts.

Superwise supports deploying, serving, monitoring, or operating AI and machine-learning systems.

systemprompt.io supports deploying, serving, monitoring, or operating AI and machine-learning systems.

TeamoRouter provides machine-learning models, research, training resources, or evaluation tools.

Tecton supports deploying, serving, monitoring, or operating AI and machine-learning systems.

TensorRT-LLM

github.com

Inference engine for TensorRT on Nvidia GPUs.

TensorZero builds open-source tools for production-grade LLM applications: LLM gateway, observability, optimization, evaluations, and experimentation.

Inference for text-embedding models.

text-generation-inference

github.com

Large Language Model Text Generation Inference.