Directory
Published directory entries matching your search.
MovieLens 1M is a stable benchmark dataset of movie ratings containing 1 million ratings from 6000 users on 4000 movies, released in February 2003. It is available as a downloadable archive with a checksum and README.
Evals is a framework for evaluating large language models and systems built using LLMs. It offers an existing registry of evals to test different dimensions of OpenAI models and the ability to write custom evals. It is available as an open-source registry of benchmarks. Evals require an OpenAI API key.
OpenLM Arena is a leaderboard site that ranks large language models. It bases rankings on three benchmarks: an agent-driven battle platform with Bradley-Terry Elo ratings from LLM-as-a-judge votes, a Model Intelligence Index, and the ARC-AGI-2 general intelligence benchmark. The page lists models with Elo, coding, vision, MII and ARC-AGI figures alongside organization and license.
Ponytail 5 is a ruleset for AI coding agents intended to make them write the least code that works. It provides a review command and an audit command, and states tests accompany risky logic. The project reports benchmark comparisons against running without it.
PricePerToken is a free comparison site for LLM API pricing. It presents per-token costs for models from providers including OpenAI, Anthropic, Google and Mistral, alongside model listings with context windows and benchmark figures. Pages cover individual models such as Amazon Nova Lite 1.0 and Nova Micro 1.0, whose pricing and context windows are stated.
PyTorch Geometric Temporal is a temporal extension library for PyTorch Geometric. It contains dynamic and temporal geometric deep learning, embedding and spatio-temporal regression methods from published research papers. It includes a dataset loader, train-test splitter and temporal snapshot iterator for dynamic and temporal graphs, with GPU support and benchmark datasets.
LLMeBench is a flexible framework for accelerating the benchmarking of large language models. It provides tools to run benchmarks and evaluate model performance. The project is hosted on GitHub under the qcri organization.
CLI tool for debugging and benchmarking RAG retrieval. EXPLAIN ANALYZE for your retrieval layer.
RunThisLLM is a website that provides hardware requirements for running local AI models. It lists over 295 models with full build specifications, performance estimates, and benchmarks. It also offers a Chrome extension and a custom machine building service.
WhisperS2T is an optimized speech-to-text pipeline for the Whisper model that supports multiple inference engines. The repository provides release notes, quickstart guides, future roadmaps, and a benchmark and technical report.
SimpleBench is a multiple-choice text benchmark for LLMs with over 200 questions covering spatio-temporal reasoning, social intelligence and trick questions. Its report states a non-specialized human baseline of 83.7 percent from a small sample of nine participants.
SWE-bench is a benchmark for evaluating large language models on real-world software issues collected from GitHub. Given a codebase and an issue, a language model generates a patch that resolves the described problem.
The Pile is an 825 GiB open source language modelling data set made of 22 smaller high-quality datasets combined together. It is offered as a training set and as a benchmark, and is distributed as jsonlines data compressed with zstandard.
A visualisation of major large-language models ranked by performance using the MMLU benchmark, which consists of 16,000 multiple-choice questions across 57 academic subjects. It also presents selected lawsuits filed against AI companies and notes that MMLU has some critiques.
torchvision is a package of popular datasets, model architectures and common image transformations for computer vision. The repository holds documentation, benchmarks, examples, packaging, references and tests alongside the torchvision code. Installation instructions point to official guidance for stable torch and torchvision versions; building from source is covered on the contributing page.
TruLens instruments your AI agent with OpenTelemetry, scores every step with benchmarked LLM judges, and tells you which version to ship.
VBench is a Hugging Face Space created by Vchitect that hosts a leaderboard for AI video generation models. Users can upload a zip file with their model's evaluation JSON files and provide details such as model name, link, team, and contact email. The application extracts the metrics and updates the benchmark leaderboard accordingly.
WhichLLM is a tool that identifies local large language models that run effectively and perform best on a user's hardware. It ranks models by real, recency-aware benchmarks rather than parameter count, and can be run instantly with a single command.
WizardLM is a repository of large language models including WizardLM, WizardCoder and WizardMath, described as LLMs built upon Evol Instruct. The page lists model checkpoints and benchmark results, with WizardCoder-33B-V1.1 reported at 79.9 pass@1 on HumanEval and WizardMath-7B-V1.1 reported at 83.2 pass@1 on GSM8k.
The Wolfram LLM Benchmarking Project is an ongoing initiative that tracks large language model performance using a benchmark centered on Wolfram Language code generation tasks. It publishes the results of its ongoing performance tracking for LLMs.
A public archive of acceleration techniques for the Stable Diffusion model intended to help generate images faster and more efficiently. It covers AITemplate, nvFuser, TensorRT and FlashAttention, with example images and benchmarks. The repository was archived by the owner on Oct 24, 2025, and is now read-only.
