TorchTitan

PyTorch reference codebase for pre-training LLMs from scratch with FSDP2 and DTensor.