The Pile

The Pile is an 825 GiB open source language modelling data set made of 22 smaller high-quality datasets combined together. It is offered as a training set and as a benchmark, and is distributed as jsonlines data compressed with zstandard.

Website Directory import
0 shouts Added Sep 23, 2026 pile.eleuther.ai
pile.eleuther.ai/
The Pile website preview

Features

Links

Tags