Directory
Published directory entries matching your search.
Modelz LLM is an inference server for running open source large language models such as FastChat, LLaMA and ChatGLM on local or cloud environments with an OpenAI compatible API. It supports interaction through the OpenAI python SDK or LangChain and provides docker images for deployment on Kubernetes or Modelz.
Machine learning model serving framework with dynamic batching and pipelined stages, provides an easy-to-use Python interface.
Msty describes private AI for individuals, teams and organizations, with four products connected as one system: Studio, a private AI workspace; Go, for agents; Nexus, a model gateway hub; and Stack, for organizing knowledge. Studio pricing lists a free tier and an Aurum plan billed yearly at $149 per user.
MyVibe is a tool that uses AI to transform web projects into deployable applications. It allows users to upload, convert, and publish their web creations instantly, and is powered by ArcBlock.
NASGym is a proof-of-concept OpenAI Gym environment for Neural Architecture Search. It implements reinforcement learning steps for NAS and encodes networks using the Neural Structure Code from BlockQNN. A network is modeled as a list of NSCs, an action adds a layer, and the reward is accuracy after early-stop training. CIFAR-10 is the default dataset. The code and documentation may contain minor bugs.
NCNN is a neural network inference framework optimized for mobile, embedded, and desktop deployment. It has no third-party runtime dependencies and runs across CPU and Vulkan GPU backends. Tools such as pnnx convert PyTorch and ONNX models to ncnn, allowing deployment to phones, PCs, browsers, and edge devices. The project is used in several Tencent applications.
NDScala is a Scala 3 library for N-dimensional and multi-dimensional arrays (tensors), described as comparable to NumPy ndarray or PyTorch Tensor, with type safety over shapes, array and axis labels and numeric data types. The repository includes materials on shape-safe neural network training and an ONNXScala component.
Easy-to-use library to boost AI inference.
Nimblebox supports deploying, serving, monitoring, or operating AI and machine-learning systems.
An Easy-to-Use and High-Performance AI deployment framework.
Neural networks framework in pure C: training and inference, no dependencies.
Serve Llama 2 and other large language models locally from command line or through a browser interface.
Ombre is an open-source AI infrastructure layer. It runs eight specialized agents on every request to handle security, caching, memory, hallucination detection, and tamper-proof audit trails. It is installed with one line of code, and user data never leaves the user's server.
OneComp is a Python package for large language model (LLM) compression, hosted as a public repository under the FujitsuResearch GitHub organization. The project is designed for LLM compression tasks and accepts community contributions to its codebase.
onWatch is an open-source tool developed by onllm-dev that tracks AI API quotas for services including Synthetic, Z.ai, Anthropic Claude Code, Codex, GitHub Copilot and Antigravity in real time. It functions as a lightweight background daemon that uses less than 50MB of RAM, stores data in SQLite, and provides a Material Design 3 dashboard. The tool does not collect telemetry.
Open-source project that supports deploying, serving, monitoring, or operating AI and machine-learning systems.
OpenLIT is an open-source agent harness engineering platform built on OpenTelemetry. It provides tracing, evaluations, guardrails, prompt and context management, and cost and GPU monitoring for the harness around AI agents and coding agents. It is free to self-host under Apache 2.0 and is described as working with many models, frameworks and harnesses.
Jan is an open source alternative to ChatGPT that runs offline on the user's computer, according to the repository description. It provides local AI models downloaded and run with an OpenAI-compatible local server at localhost:1337 and supports connecting to cloud GPT and Claude models according to the README features list.
Turns your ML code into microservices with web API, interactive GUI, and more.
Platform and SDK for AI Engineers providing tools for LLM evaluation, observability, and a version-controlled enhanced prompt playground.
pgvecto.rs is an open-source tool that provides scalable, low-latency, hybrid-enabled vector search for PostgreSQL. It is designed to enhance vector search capabilities without modifying the core database system, per its official GitHub repository.
Open-source vector similarity search for Postgres.
Portkey is a production stack for generative AI builders that offers a suite of tools including an AI gateway, observability features, guardrails and prompt management modules. It is designed to help organizations democratize and productionize generative AI.
A high-speed inference engine for deploying LLMs locally.
