Directory
Published directory entries matching your search.
Groq is a platform for inference, described as one fully integrated platform for infrastructure, inference, and control. It offers tiers including GroqMetal for bare-metal infrastructure, GroqCore for an inference stack, and GroqAssured for governance, auditability, and control.
A 2003 book by David J.C. MacKay published by Cambridge University Press, covering information theory, inference, and learning algorithms. The site offers corrections, software, and downloadable figures. It lists a price of £35.00 or $60.00.
Kaito is the Kubernetes AI Toolchain Operator, an operator suite that automates LLM inference, fine-tuning and RAG engine deployment in a Kubernetes cluster. It simplifies the CRD API and provides optimized preset configurations for inference engine scheduling parameters. Workspaces are the basic CRD for managing inference and tuning workloads, and all vLLM-supported models can be run.
Triton Inference Server is open source inference serving software that streamlines AI inferencing. It enables teams to deploy models from multiple deep learning and machine learning frameworks across cloud, data center, edge and embedded devices.
bitnet.cpp is the official inference framework for 1-bit LLMs such as BitNet b1.58. It provides optimized kernels for inference of 1.58-bit models on CPU and GPU, with NPU support stated as coming next. The repository includes documentation and setup scripts, and the project is released under the MIT license.
DigitalOcean describes an AI-native cloud platform with a unified stack for agents, inference and cloud infrastructure. Products listed include Managed Agents, Learning Services, Data Services, an Inference Engine with over 75 models, and core cloud services such as Droplets, Kubernetes and App Platform. Harness Runtime is listed starting at $0.0535/hr for durable cloud sandboxes for AI coding agents.
Flux is the official inference repository for FLUX.1 models from Black Forest Labs. It contains minimal inference code for running image generation and editing with the open-weight models, including text-to-image, in/out-painting, structural conditioning and image variation variants. The repository is distributed under the Apache-2.0 license.
Forefront is described as a platform to fine-tune and inference open-source language models. The site mentions building with open-source AI, fine-tuning models on data, and running inference through serverless API endpoints. It also describes storing training, validation and evaluation data in ready-to-fine-tune datasets. Pricing lists a free tier and a team plan at $99 per month, each including $20 in free credits.
Host inference APIs, bulk inference and fine tune text, vision, audio and multi-modal models.
GPyTorch is a Gaussian process library implemented using PyTorch, designed for creating scalable, flexible, and modular Gaussian process models. It performs most inference operations using numerical linear algebra techniques such as preconditioned conjugate gradients, and provides GPU acceleration through matrix-vector-multiplication-based inference. Installation requires Python 3.10 or later, PyTorch 2.4.1 or later, and NumPy 2.0 or later.
layer is a command-line tool for neural network inference, following the Unix philosophy. It enables users to perform inference from the command line. The project is open-source and hosted on GitHub.
libonnx is a lightweight, portable pure C99 ONNX inference engine for embedded devices with hardware acceleration support. Its .c and .h files can be dropped into a project and compiled with it. Models are loaded from a file into a context, and inference runs on input tensors via onnx_run. It is based on ONNX version v1.17.0 with opset 24 support, and some operators are not implemented.
LMFlow is a toolkit for finetuning and inference of large foundation models, presented as extensible, convenient and efficient. It offers support for custom optimizer training, conversation templates, speculative decoding, long context inference with position interpolation for LLaMA models, and Flash Attention-2. The project is released under the Apache-2.0 license and was made available on PyPI.
Optimum-TPU is a Hugging Face project providing Google TPU optimizations for transformers models, aiming to ease training and inference on TPUs. It supports text generation models including Gemma, Llama2, Llama3 and Mistral, and includes a text-generation-inference backend. The repository was archived in January 2026 and is read-only, with the project in maintenance mode.
PyVarInf is a Python package that facilitates the use of Bayesian deep learning methods with variational inference for PyTorch, providing facilities to train PyTorch neural network models using variational inference. It approximates the posterior distribution over network weights with a parametric distribution.
Seeing Theory is an archived site offering a visual introduction to probability and statistics through interactive visualizations. It covers six chapters: basic probability, compound probability, probability distributions, frequentist inference, Bayesian inference and regression analysis. It was created by Daniel Kunin while an undergraduate at Brown University.
A toolkit for deploying and serving open source text embeddings and sequence classification models. It supports models including FlagEmbedding, Ember, GTE and E5, and features token based dynamic batching, Flash Attention and Candle optimized inference, Safetensors and ONNX weight loading, and Metal support for local execution on Macs.
Text Generation Inference is a toolkit for deploying and serving large language models, implemented as a Rust, Python and gRPC server. It supports models including Llama, Falcon, StarCoder, BLOOM and GPT-NeoX, with features such as token streaming, continuous batching and quantization. The repository is archived, read-only, and in maintenance mode, with pull requests limited to minor bug fixes and maintenance.
A C/C++ port of OpenAI's Whisper automatic speech recognition model, providing high-performance inference without dependencies. It supports CPU-only inference and GPU backends, and runs on platforms including macOS, iOS, Android, Linux, Windows, WebAssembly and Raspberry Pi. The whisper-cli example currently runs only with 16-bit WAV files.
Docker is a platform for building, sharing, and running containerized applications. It provides tools for AI agent sandboxing, agent governance, secure software supply chain management, containerization, local LLM inference, and enterprise-grade application security. The platform is designed to simplify development setup, support local to scaled deployment, and avoid vendor lock-in.
19 Questions is an open source machine learning and Bayesian inference engine designed to assign attributes to objects. Its source code is hosted in a public GitHub repository under the fulldecent organization, and contributions to the project are accepted from the public.
Adaptive Reliability Layer is a Python project that sits between an inference pipeline and a monitoring layer. It detects distribution shift, decides whether adaptation is warranted, and learns from delayed revealed labels. It does not replace the model.
A personal blog by Alex Smola titled Adventures in Data Land, covering machine learning topics. Entries include efficient LLM inference tutorials, random projections, effective sample size, voice agents and evaluation, and porting MXNet to Blackwell and Apple Silicon. Posts reference events such as the Machine Learning Summer School 2026 in NYC.
Aerosolve is a machine learning package built on the JVM. It provides features such as geo-based features, controllable quantization and feature interaction, and a feature transform language that gives developers control over features. General additive models are used. Scala handles feature training, with a lightweight Java API for run time inference.
