Sophia Optimizer

Second-order optimizer that pre-trains LLMs 2× faster than AdamW — open reference impl.