Nanotron

HuggingFace minimal 3D-parallelism library for pretraining LLMs from scratch.