GaLore Training

Gradient low-rank projection lets you pre-train 7B LLMs on a single consumer GPU.