Directory
Published directory entries matching your search.
vLLM is an open-source library for efficient serving of large language models. It provides an API server, supports offline and online inference, deployment options, and features such as automatic prefix caching, batch invariance, context extension, and speculative decoding. It can be used with GPUs, CPUs, and TPUs.
WebNN is a web standard that lets web apps and frameworks accelerate deep neural networks using on-device hardware such as GPUs, CPUs, or purpose-built AI accelerators. The introduction states that in-browser inference supports local media sources, keeps user data on-device, and needs no network after initial asset caching.
A Hackaday article by Matthew Carlson explaining why LLaMa matters. It describes LLaMa as a transformer language model from Facebook/Meta research spanning 7 billion to 65 billion parameters, and notes that llama.cpp runs inference with 4-bit quantization.
Bittensor is an open, decentralized network where independent subnets produce digital commodities such as compute, inference, storage and prediction, and the chain pays contributors in TAO. Miners produce the commodity, validators score miners, subnet creators define the incentive mechanism, and stakers back validators. The Bittensor Python SDK and btcli cover user-facing chain operations.
Xinference is a model serving tool that provides a unified production-ready inference API. It allows users to run open-source, speech and multimodal models on cloud infrastructure, on-premises systems or personal laptops by modifying a single line of code.
xTuring is an open-source project for building, personalizing, and controlling LLMs, covering data pre-processing through fine-tuning. It includes a simple API for data prep, training, and inference, LoRA and low-precision (INT8/INT4) support, and built-in evaluation metrics such as perplexity. Models can run locally or in a private VPC and scale from CPU to multi-GPU.
PyTorch implementation of YOLOv3, YOLOv3-SPP, and YOLOv3-tiny for real-time object detection. The repository includes training, validation, inference, and multi-format export tooling, and reuses shared utilities from the ultralytics package. Detection is framed as a single regression problem predicting bounding boxes and class probabilities from full images in one forward pass.
