Directory
Published directory entries matching your search.
ANEE is an experimental dynamic inference wrapper for pretrained Transformer language models, currently GPT-2. It exposes an energy_budget and performs early exit inside the model's forward pass rather than always running all layers. It adds per-token adaptive computation using a profiler and learned controller to skip redundant layers while keeping the KV-cache aligned.
BudgetML is a tool for deploying a machine learning inference service to a secured HTTPS API endpoint with less than 10 lines of code. It runs the model on a Google Cloud Platform preemptible instance, generates a FastAPI server endpoint, and provides interactive Swagger docs. The project states that the library is not being actively maintained.
A personal blog about AI system design and bringing AI into production. Posts cover topics including generative AI applications, agents, multimodal models, LLM research challenges, model routing, generation configurations, and machine learning systems.
Clip-as-a-service is an open-source tool that provides scalable embedding, reasoning, and ranking capabilities for images and text using the CLIP model. It is maintained as a public repository on GitHub under the jina-ai organization, with documentation and installation guides available.
Cog is an open-source tool that packages machine learning models in standard, production-ready containers. Users define the environment in a configuration file, and it generates a Docker image with Nvidia base images and dependency caching. Model inputs and outputs are defined in Python, from which Cog generates an OpenAPI schema and an HTTP inference server. Models can be deployed to own infrastructure or Replicate.
Dify is a platform for building agentic workflows on one canvas, covering agents, knowledge pipelines, models and tools. Deployments include Dify Cloud, a VPC or self-hosted setup, and a source-available Community Edition. The platform also supports RAG pipelines with AI model and tool support.
FACTORIE is an open-source toolkit for deployable probabilistic modeling, implemented as a software library in Scala and hosted on GitHub under the factorie organization. It provides users with a succinct language for creating relational factor graphs, estimating model parameters, and performing inference, with documentation for installation, example usage, and pre-built NLP models.
FeatherCNN is a lightweight CNN inference library developed by Tencent AI Platform Department, originating from a game AI project for King of Glory. It targets ARM CPUs and is developed for mobile phones, embedded devices and ARM-based servers. It accepts Caffe models, which are converted into a single binary model.
Ultravox is a real-time voice AI infrastructure layer for building voice agents. It trains a speech-native model and manages its own inference and infrastructure stack. The site offers developer-friendly APIs and agentic-ready primitives intended for natural, conversational voice agents. Ultravox has been acquired by Inworld. Pricing lists pay-as-you-go at $0 per month with the first 30 minutes free and $0.05 per minute after that.
Go binding for MXNet c predict api to do inference with a pre-trained model.
Guild is a vendor-neutral control plane for AI agents. It lets engineering teams manage, govern, and share agents in production, with security and observability built in. Features include scoped credentials, action approval, policy enforcement, tracing, and evaluations. It supports any model or framework and provides a shared agent catalog and CLI access.
Build multimodal AI services via cloud native technologies · Model Serving · Generative AI · Neural Search · Cloud Native.
Kitaru is an open-source, self-hosted tool providing replay-based evaluations for AI agents. It records or imports production runs as sessions and replays them against a changed model, prompt, or code, with tool calls answered from the recording. It is available for Python and TypeScript and comes from the team behind ZenML.
Open source MLOps project that eases model handoffs between data scientist and DevOps.
Machine learning model serving framework with dynamic batching and pipelined stages, provides an easy-to-use Python interface.
Msty describes private AI for individuals, teams and organizations, with four products connected as one system: Studio, a private AI workspace; Go, for agents; Nexus, a model gateway hub; and Stack, for organizing knowledge. Studio pricing lists a free tier and an Aurum plan billed yearly at $149 per user.
Pipeline Conversations is a podcast from ZenML featuring interviews with platform engineers, ML practitioners and technical leaders building production AI systems. Episodes cover MLOps and LLMOps topics including Kubernetes workflows, model fine-tuning and evaluation, LLM security, cost optimization, prompt engineering and agents in production.
Rapid-MLX is an open-source, Apache 2.0 licensed large language model inference server and Mac application for Apple Silicon devices. It is compatible with OpenAI and Anthropic APIs, built on the MLX framework, and focused on reliable tool calling for coding agents.
Python-free Rust inference server with OpenAI API compatibility and hot model swapping.
An MLOps/LLMOps platform for model building, evaluation, and fine-tuning.
Weco is a research and product lab building recursively self-improving AI. Its platform points at an eval script and iteratively optimizes code against a metric the user defines, such as speed, accuracy or cost. Supported model providers include GPT, Claude, Gemini, Kimi, GLM and MiniMax, with bring your own key.
Whylogs is an open-source data logging library for machine learning models and data pipelines. It provides visibility into data quality and model performance over time, and supports privacy-preserving data collection to support system safety and robustness.
A guide to MLOps is an open-source tutorial from the Swiss AI Center team that walks engineers and small teams from notebook experiments to production. It covers local training and model evaluation, moving the model to the cloud, serving and deploying, monitoring and maintaining, and labeling data and retraining, using tools such as DVC, Git, GitHub Actions, CML, Docker, BentoML, Fluent Bit, Evidently AI, and Label Studio.
Agenta is an open-source workspace for AI agents. Agents are built through chat, improved with feedback, and shared with a team, either self-hosted or in the cloud. Integrations and messaging in Slack or Telegram are described on the site.
