BitsExplorer

BitsExplorer finds what people are building with AI, every day. See today's picks →

Preview of Era V5 S11 Optimizers

Era V5 S11 Optimizers

ERA V5 Session 11: optimizer claims re-measured on a char-level GPT. Adam by hand, bias correction, per-layer update ratios under warmup, cosine vs WSD, LR transfer across width under SP and muP, and a tuned AdamW vs Muon comparison.

Description excerpted from the original listing, which is linked below.

Open it Source code

The week in AI-built software

Every Monday: how many projects were built with AI that week, which tools built them, where they came from, and the week's picks.

Read the latest week