Add reproducible single-GPU sub-one-pass optimizer benchmark

[VinZch]

What changed

• adds an exact token-budgeted sub-one-pass protocol with fixed sequence boundaries and deterministic no-replacement sampling
• integrates all 20 existing optimizers into one sequential single-GPU runner
• adds FineWeb token-bin preparation, LR selection, server pipeline, S2 main curves, and S3 paired-seed confirmation
• fixes the muon-pytorch parameter split so matrix parameters use Muon while embeddings, norms, and the head use AdamW
• includes lightweight formal manifests, summaries, logs, and the aggregate CSV; datasets and checkpoints are intentionally excluded

Experiment

The formal run used a 33.48M-parameter Llama model, 65,536 tokens per step, and evaluation at 16M, 32M, 64M, 128M, and 256M observed tokens on a fixed 512M-token FineWeb corpus. S2 completed all 20 optimizers. S3 confirmed AdamW plus the three leading non-AdamW optimizers using seeds 1 and 2.

S3 mean final validation loss across confirmation seeds:

• AdEMAMix: 4.0175
• SOAP: 4.0372
• Muon: 4.0440
• AdamW: 4.1034

Validation

• python3 -m pytest -q: 14 passed
• downloaded lightweight results were checksum-verified against the server copy
• S2 manifest: 20 completed, 0 failed
• S3 manifest: 8 completed, 0 failed