Datatrove

HuggingFace scalable data-processing library for creating large LLM training datasets.