RedPajama-Data-v2 by Together AI

RedPajama-Data-v2 is a 30 trillion token open dataset from 84 CommonCrawl dumps across five languages, with over 40 precomputed quality annotations. Data processing scripts are open source on GitHub and all data are available on HuggingFace. The dataset is intended for further filtering and weighting of data, and the release notes describe it as an open dataset for training large language models.

Website Directory import
0 shouts Added Sep 23, 2026 together.ai
www.together.ai/blog/redpajama-data-v2
RedPajama-Data-v2 by Together AI website preview

Features

  • 30 trillion token open dataset from 84 CommonCrawl dumps across five languages Source: www.together.ai

Links

Tags