github.com
Open-source project that supports deploying, serving, monitoring, or operating AI and machine-learning systems.
Directory
AI Infrastructure & MLOps
Explore AI Infrastructure & MLOps resources in AI & Machine Learning.
github.com
MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
dify.ai
Dify is a platform for building agentic workflows on one canvas, covering agents, knowledge pipelines, models and tools. Deployments include Dify Cloud, a VPC or self-hosted setup, and a source-available Community Edition. The platform also supports RAG pipelines with AI model and tool support.
blog.dominodatalab.com
Dominodatalab supports deploying, serving, monitoring, or operating AI and machine-learning systems.
edwardlib.org
Edward is a Python library for probabilistic modeling, inference, and criticism. It is a testbed for experimentation and research with probabilistic models, from hierarchical models on small data sets to deep probabilistic models on large data sets. It fuses Bayesian statistics and machine learning, deep learning, and probabilistic programming, and is built on TensorFlow.
github.com
Epsilla is a high-performance vector database hosted on GitHub under the epsilla-cloud organization. It leverages parallel graph computing for its operations, and offers an experimental option to use it as a Python library without requiring a Docker image, alongside Epsilla Cloud as a deployment option.
github.com
Eurybia is an open-source tool hosted on GitHub under the MAIF organization. It is designed to monitor model drift over time and secure model deployment processes through data validation functionality, with public documentation covering overview, quickstart, installation, features, and project background.
github.com
Evidently is an open-source observability framework for machine learning and large language models, hosted on GitHub under the evidentlyai organization. It enables users to evaluate, test, and monitor AI-powered systems and data pipelines, with support for use cases ranging from tabular data processing to generative AI, and includes more than 100 built-in metrics.
github.com
Faiss is a library for efficient similarity search and clustering of dense vectors, containing algorithms that search sets of vectors of any size. It is written in C++ with complete wrappers for Python/numpy, with some algorithms implemented on the GPU, and is developed primarily at Meta's Fundamental AI Research group.
github.com
FACTORIE is an open-source toolkit for deployable probabilistic modeling, implemented as a software library in Scala and hosted on GitHub under the factorie organization. It provides users with a succinct language for creating relational factor graphs, estimating model parameters, and performing inference, with documentation for installation, example usage, and pre-built NLP models.
github.com
FeatherCNN is a lightweight CNN inference library developed by Tencent AI Platform Department, originating from a game AI project for King of Glory. It targets ARM CPUs and is developed for mobile phones, embedded devices and ARM-based servers. It accepts Caffe models, which are converted into a single binary model.
github.com
An enterprise-grade, high performance feature store.
github.com
Featureform is an open-source virtual feature store hosted on GitHub under the featureform organization. It is designed to turn users' existing data infrastructure into a feature store, with supporting resources including a public website, documentation, community forum, and explanatory material on its core abstraction and unique value proposition.
fixie.ai
Ultravox is a real-time voice AI infrastructure layer that powers fast, natural, and scalable voice agents.
ultravox.ai
Ultravox is a real-time voice AI infrastructure layer for building voice agents. It trains a speech-native model and manages its own inference and infrastructure stack. The site offers developer-friendly APIs and agentic-ready primitives intended for natural, conversational voice agents. Ultravox has been acquired by Inworld. Pricing lists pay-as-you-go at $0 per month with the first 30 minutes free and $0.05 per minute after that.
github.com
Future AGI is an open-source, self-hostable platform for evaluating, observing, and improving LLM and AI agent applications. It combines tracing, evals, simulations, datasets, a gateway and guardrails on one platform with a feedback loop. The Apache 2.0 core can be self-hosted, and a managed Cloud option is offered.
github.com
Future AGI SDK is a production-grade software development kit for AI evaluation, prompt management, and observability. It offers automated evaluations with sub-100ms guardrails and does not require human-in-the-loop. The SDK supports Python and TypeScript.
github.com
Go binding for MXNet c predict api to do inference with a pre-trained model.
gpuperhour.com
GPU Per Hour is a website that compares live cloud GPU pricing from 31 providers. It lists prices per GPU-hour for various GPU models, including H100, H200, B200, A100, and RTX 4090, and provides market snapshots and historical averages.
github.com
Gradio is an open-source Python library that enables developers to build and share machine learning web applications easily. It provides simple interfaces for creating demos and sharing models, supporting a wide range of ML frameworks.
oracle.github.io
GraphPipe is a protocol and collection of software designed to simplify machine learning model deployment. It decouples deployment from framework-specific model implementations, allowing models to be served in a consistent manner across different environments.
groq.com
Groq is a platform for inference, described as one fully integrated platform for infrastructure, inference, and control. It offers tiers including GroqMetal for bare-metal infrastructure, GroqCore for an inference stack, and GroqAssured for governance, auditability, and control.
guild.ai
Guild is a vendor-neutral control plane for AI agents. It lets engineering teams manage, govern, and share agents in production, with security and observability built in. Features include scoped credentials, action approval, policy enforcement, tracing, and evaluations. It supports any model or framework and provides a shared agent catalog and CLI access.
helicone.ai
Helicone AI supports deploying, serving, monitoring, or operating AI and machine-learning systems.