The Pile is an 825 GiB open source language modelling data set made of 22 smaller high-quality datasets combined together. It is offered as a training set and as a benchmark, and is distributed as jsonlines data compressed with zstandard.
Marquez
Marquez is an open source metadata service for collecting, aggregating and visualizing a data ecosystem's metadata. It maintains provenance of how datasets are consumed and produced and provides visibility into job runtime and dataset access. Metadata is collected via OpenLineage, and a web UI shows job and dataset dependencies through a lineage graph.
Features
- collect, aggregate, and visualize a data ecosystem's metadata Source: github.com
- maintains the provenance of how datasets are consumed and produced Source: github.com
Links
Tags
-
0The Pile is an 825 GiB open source language modelling data set made of 22 smaller high-quality datasets combined together. It is offered as a training set and as a benchmark, and is distributed as jsonlines data compressed with zstandard.
-
0pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language. It is described as a data analysis and manipulation tool built on top of Python. Documentation, a user guide and an API reference are available.
-
0
Exhibit is an open source publishing framework for creating data-rich interactive web pages. It provides advanced text search and filtering, along with interactive maps, timelines and other visualizations. The software is licensed under the BSD license and was originally sponsored partially by The Andrew W. Mellon Foundation as part of the SIMILE project.
-
0Bricks is an AI-powered workspace for organizing data, building dashboards, creating slides, and collaborating. It generates interactive dashboards from CSV and Excel data, and its documentation lists spreadsheets, live-updating dashboards, slides, and real-time collaboration.
-
0
LochBot is a free prompt injection vulnerability checker. It analyzes a pasted system prompt against 31 documented injection attack patterns in 7 categories and returns a security score with fix suggestions. The analysis runs client-side in the browser. Results can be exported as JSON.
-
0The Analytics Engineering Roundup is a Substack newsletter curating articles on analytics engineering and its adjacent ecosystem, curated by Tristan Handy. He describes analytics engineering as creating and disseminating knowledge within organizations, covering cloud data platforms, dimensional modeling and data quality.
Community notes
Reviews and discussion
No reviews yet
Be the first to share how this project worked for you.
Join the discussion
Sign in with a verified account to leave a review or join the discussion.
Sign in to review