Search results

LLM

llm.datasette.io

LLM provides machine-learning models, research, training resources, or evaluation tools.

Llm

github.com

Open-source project that provides machine-learning models, research, training resources, or evaluation tools.

Aider LLM Leaderboards

aider.chat

Aider LLM Leaderboards presents quantitative benchmarks of LLM code editing skill. The polyglot benchmark tests LLMs on 225 Exercism coding exercises in C++, Go, Java, JavaScript, Python and Rust, and benchmarks evaluate an LLM's ability to follow instructions and edit code without human intervention.

confident-ai/deepeval

github.com

DeepEval is an open-source LLM evaluation framework, described as similar to Pytest but specialized for unit testing LLM apps. It provides evaluation metrics such as G-Eval, task completion, answer relevancy and hallucination, which use LLM-as-a-judge and other NLP models running locally. It can evaluate LLM apps end-to-end, complete agent trajectories, and individual agent steps.

LLM Evaluation Metrics: Everything You Need for LLM Evaluation - Confident AI

confident-ai.com

This is a blog post from Confident AI that serves as a guide to LLM evaluation metrics. It covers retrieval-augmented generation (RAG) metrics such as faithfulness and answer relevancy, agent metrics, LLM-as-a-judge approaches, and includes working DeepEval code examples.

The architecture of today's LLM applications - The GitHub Blog

github.blog

A GitHub Blog post by Nicole Choi, based on discussions with Alireza Goudarzi and Albert Ziegler, about the emerging architecture of LLM applications. It covers five steps to building an LLM app, including focusing on a single problem and choosing the right LLM, and notes that LLM outputs are probabilistic and not predictable.

briland/LLM-security-and-privacy

github.com

A curated GitHub list of papers and tools covering LLM threats and vulnerabilities from a security and privacy standpoint. It includes paper summaries, key takeaway points, citations in a bib file, and frameworks and taxonomies such as OWASP Top 10 for LLM Applications.

Falcon LLM - Home

falconllm.tii.ae

Falcon LLM is described as a generative large language model that supports applications and use cases. The site presents models in the Falcon LLM family, including Falcon-H1 and Falcon-E, and lists Falcon-H1 model insights. It also references Falcon Arabic for Modern Standard Arabic and dialects, and names Falcon-Emirati, Falcon-ASR and Falcon-OCR-Arabic.

Llm Stats

x.com

LLM Stats is an independent AI evaluations lab that measures performance on real-world tasks, with its profile pointing to llm-stats.com. The X account has 406 posts and was joined in February 2025.

AgentBench: Evaluating LLMs as Agents

huggingface.co

AgentBench is a multi-dimensional benchmark that consists of 8 distinct environments for evaluating LLMs as autonomous agents in interactive tasks. It assesses reasoning and decision-making abilities in multi-turn open-ended generation, and tests over 25 LLMs including APIs and open-source models.

Complete Guide to LLM Agents (2025)

botpress.com

A Botpress-published guide that explains large language model (LLM) agents, including their definition, core features, common types, and improvement techniques, and notes that LLM agents are AI systems for natural language tasks such as customer support and data analysis.

Free LLM API Resources

github.com

Free LLM API Resources is a GitHub repository or organization with source code, releases, documentation, or project resources.

Indico LLM Leaderboard

indicodata.ai

The Indico LLM Leaderboard provides benchmarking for large language models on extractive, classification, and predictive tasks. It includes accuracy comparisons, cost of ownership assessments, and datasets such as CORD and Kleister NDA, with the latest update in August 2024.

LLM Course

github.com

LLM Course is a GitHub repository or organization with source code, releases, documentation, or project resources.

Documentation for LLM Evaluation Clarifai Guide, covering setup, features, and practical usage.

LLM Model VRAM Calculator

huggingface.co

LLM Model VRAM Calculator is a Hugging Face Space created by NyxKrage. The tool allows users to input a Hugging Face model name, select a target GPU, set context length and quantization options, then calculates the model's required memory size and context buffer capacity to aid LLM deployment planning.

LLM Optimizer applies AI to data analysis, extraction, visualization, or business intelligence.

LLM Visualization

bbycroft.net

LLM Visualization is an AI chat or model resource for writing, coding, research, prompts, and assistant workflows.

LLMFlows

github.com

LLMFlows is a Python framework for building LLM applications such as chatbots, question-answering systems and agents. It provides abstractions for using LLMs and vector stores, with explicit classes that avoid hidden prompts or LLM calls, aiming to make monitoring, maintenance and debugging easier.

manifest.build

manifest.build

manifest.build is an open source LLM router for AI agents and related tools. It connects AI agents and harnesses to any provider in seconds, supporting subscription, pay-per-token, local model and custom provider options. The tool offers features including model fallbacks, full request logging, autofix for broken provider calls, an LLM gateway for centralized provider management, and observability tools for tracking usage, reliability and costs. The software is open source, allowing users to self-host it or contribute to its development.

OWASP Top 10 for LLM Applications

genai.owasp.org

OWASP Top 10 for LLM Applications is a security resource published by the OWASP Gen AI Security Project that outlines the 2025 top 10 risks, vulnerabilities, and corresponding mitigations for developing, deploying, and managing large language model and generative AI applications.

PAIR-code/llm-comparator

github.com

LLM Comparator is an interactive data visualization tool for evaluating and analyzing large language model responses side-by-side, developed by the PAIR team. It includes a Python library for creating JSON files and a defined JSON data format.

Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)

magazine.sebastianraschka.com

An Ahead of AI article by Sebastian Raschka sharing lessons from hundreds of experiments on finetuning LLMs with low-rank adaptation (LoRA), billed as an essential technique for efficiently training custom open-source LLMs. It covers LoRA consistency, QLoRA compute-memory trade-offs, learning rate schedulers, Adam versus SGD, multiple epochs, enabling LoRA for more layers, and balancing rank and alpha.

RasaGPT

github.com

RasaGPT is described as a headless LLM chatbot platform built on top of Rasa and Langchain. It is boilerplate and a reference implementation of Rasa and Telegram utilizing an LLM library like Langchain for indexing, retrieval and context injection.