Directory
Published directory entries matching your search.
MeTA is a modern C++ data sciences toolkit. It provides text tokenization including deep semantic features like parse trees, inverted and forward indexes with compression and caching strategies, ranking functions for searching indexes, topic models, classification algorithms, graph algorithms, language models, a CRF implementation for POS-tagging and shallow parsing, and wrappers for liblinear and libsvm.
CogCompNLP is a collection of core natural language processing libraries developed by the Cognitive Computation Group. Its modules include a lemmatizer, part-of-speech tagger, named entity recognizer, tokenizer, transliterator, relation extraction, similarity, temporal normalizer and verb sense tools. The project includes a pipeline for annotating raw text and recommends JDK8.
Bread Dataset Viewer is a VS Code extension for viewing and handling large datasets. It opens JSONL, Parquet, CSV, TSV, JSON, and Arrow/Feather files by streaming and lazy-loading data, and provides Cards, Table, and Raw views. It includes 16 production LLM tokenizers from GPT, Claude, Llama, and Qwen for chat completion data.
age is a file encryption tool, format and Go library with small explicit keys, post-quantum support, no config options, and UNIX-style composability. It provides an age-keygen command for generating keys and can encrypt or decrypt files, including piping data through it. Install packages are listed for several systems, and plugins such as age-plugin-yubikey support hardware tokens.
Compromise is a JavaScript natural-language processing library that turns text into data, offering tokenization of words, sentences and punctuation along with part-of-speech tagging, matching, and transformations such as verb conjugation and number handling. It runs on the client side, is about 250kb minified, and includes plugins, with multilingual support listed for French, German, Italian and Spanish.
GDM Science Skills to speed up agentic scientific workflows with better grounding and higher token efficiency. Integrate insights from AlphaGenome, AFDB, UniProt and 30+ other databases and tools.
