Directory
Published directory entries matching your search.
System Designer is a learning site offering interactive lessons and case studies on system design. Its Machine Learning Systems section covers building production ML systems, from data pipelines to deployment, with topics such as distributed training, model serving, MLOps monitoring and A/B testing. Lessons are organized into units with stated difficulties and durations.
Self-hosted control plane that sits between connected AI clients, model providers, and tools. It applies access rules to users and agents, routes supported model requests through a gateway, checks governed tool calls before execution, and records policy decisions, execution records, and usage in the operator's environment. It deploys a Rust runtime with PostgreSQL on infrastructure the operator controls.
TeamoRouter provides machine-learning models, research, training resources, or evaluation tools.
Tecton supports deploying, serving, monitoring, or operating AI and machine-learning systems.
TensorFlow Model Optimization is a toolkit designed to optimize machine learning models for deployment for Keras and TensorFlow. It includes techniques such as quantization and pruning to reduce model size and improve inference performance.
Inference engine for TensorRT on Nvidia GPUs.
TensorZero builds open-source tools for production-grade LLM applications, including an LLM gateway, observability, optimization, evaluations, and experimentation. The project remains available on GitHub but is no longer maintained. Its website is tensorzero.com.
A toolkit for deploying and serving open source text embeddings and sequence classification models. It supports models including FlagEmbedding, Ember, GTE and E5, and features token based dynamic batching, Flash Attention and Candle optimized inference, Safetensors and ONNX weight loading, and Metal support for local execution on Macs.
Text Generation Inference is a toolkit for deploying and serving large language models, implemented as a Rust, Python and gRPC server. It supports models including Llama, Falcon, StarCoder, BLOOM and GPT-NeoX, with features such as token streaming, continuous batching and quantization. The repository is archived, read-only, and in maintenance mode, with pull requests limited to minor bug fixes and maintenance.
Flexible, high-performance serving system for machine learning models.
TNN is a deep learning inference framework for mobile, desktop and server, developed by Tencent Youtu Lab and Guangying Lab. It is based on ncnn and Rapidnet and extends support to X86 and NV GPUs. The project states that it has been used in Tencent applications including Mobile QQ, Weishi and Pitu.
TorchServe is an open-source tool for serving, optimizing, and scaling PyTorch models in production. It offers quick start guides for TorchServe, conda, and Docker, and includes an announcement about security changes.
Traceloop OpenLLMetry is an open-source observability tool for GenAI and LLM applications, based on OpenTelemetry and maintained by Traceloop. It provides instrumentation for vector databases and frameworks, and includes documentation on getting started and supported destinations.
TrackingAI compares AI model results on two IQ tests, Mensa Norway and an Offline Test whose questions are not published. It presents charts of AI IQ scores, model rankings, answer speed, cost, question-by-question accuracy and changes over time.
Triton Inference Server is open source inference serving software that streamlines AI inferencing. It enables teams to deploy models from multiple deep learning and machine learning frameworks across cloud, data center, edge and embedded devices.
TrueFoundry is an enterprise AI gateway that sits between applications and LLM providers and MCP servers. It offers a unified API for accessing more than 1000 LLMs, along with routing, fallbacks, rate limiting, access control, budgets, guardrails and observability. Deployment options include SaaS, hybrid and self-hosted VPC environments.
Package, configure, and iterate on a model with Truss at whatever level of control your model needs.
Tune is a Python library for experiment execution and hyperparameter tuning at any scale. It supports machine learning frameworks including PyTorch, XGBoost, and TensorFlow and Keras, and runs algorithms such as Population Based Training (PBT) and HyperBand/ASHA. It also integrates with hyperparameter optimization tools including Ax, BayesOpt, BOHB, Nevergrad, and Optuna.
Vald is a highly scalable distributed vector search engine. It is hosted on GitHub and provides documentation on installation using Helm, requirements, and usage. The project focuses on delivering a scalable solution for vector search.
Vast.ai is a cloud GPU rental platform for AI, machine learning, deep learning and rendering workloads. Prices are set by supply and demand across 40+ data centers, with on-demand, interruptible and reserved instance types and per-second billing. Deployment is available through console, CLI, SDK and REST API.
Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.
VectorDB is a Python vector database designed to be simple and sufficient. It can be used locally or as a service, with server and client components. The project emphasizes ease of use for vector database needs.
Vellum is a personal AI assistant available locally or in the cloud, with long-term memory across conversations. It can send emails, manage calendars, browse the web, create slides, write code, run research, and execute recurring background tasks. It offers fine-grained permissions requiring approval for high-stakes actions until marked trusted. Two plans exist: Base (free) and Pro packages Mighty, Super, and Ultra.
Gemini Enterprise Agent Platform, formerly known as Vertex AI, is a comprehensive platform for developers to build, scale, govern, and optimize agents. It provides tools and services for agent lifecycle management.
