github.com
Inference for text-embedding models.
Directory
Search results
Published directory entries matching your search.
github.com
Large Language Model Text Generation Inference.
github.com
A machine learning / bayesian inference assigning attributes to objects.
github.com
Drop-in AsyncOpenAI replacement that transparently batches requests via the Batch API for cheaper LLM inference.
github.com
Bayesian Inference Tools in Python.
github.com
Official inference framework for 1-bit LLMs, by Microsoft. opensource.
github.com
Deploy a ML inference service on a budget in less than 10 lines of code.
github.com
An agentic company research tool powered by LangGraph and Tavily that conducts deep diligence on companies using a multi-agent framework. It leverages Google's Gemini 2.5 Flash and OpenAI's GPT-5.1 on the backend for inference.
github.com
Fast inference engine for Transformer models in C++.
github.com
MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
github.com
Fast inference engine for whisper in C++ using CTranslate2.
github.com
FeatherCNN is a high performance inference engine for convolutional neural networks.
github.com
Library for high performance deep learning inference on NVIDIA GPUs.
github.com
Benchmarks of machine learning inference for Go.
github.com
Go binding for MXNet c predict api to do inference with a pre-trained model.
github.com
Standardized Serverless ML Inference Platform on Kubernetes.
github.com
Neural network inference from the command line, implemented in CHICKEN Scheme.
github.com
A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.
github.com
Extensible Toolkit for Finetuning and Inference of Large Foundation Models.
github.com
MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.
github.com
Ncnn is a high-performance neural network inference framework optimized for the mobile platform.
github.com
Easy-to-use library to boost AI inference.
github.com
Neural networks framework in pure C: training and inference, no dependencies.
github.com
A high-speed inference engine for deploying LLMs locally.