GaLore Optimizer

Gradient Low-Rank Projection — pre-train 7B models on a single 24GB consumer GPU.