Session
Gradient Noise as a Signal: Adaptive Batch Sizing
Using the geometry of gradient noise to grow the batch size on the fly.
Speakers
The gradient-noise scale tells you how much signal a batch actually carries. We use online estimates of it to drive an adaptive batch-size schedule that starts small and grows as training stabilizes, cutting wall-clock time without hand-tuned warmup. We discuss estimator variance, failure modes at very large batches, and a reproducible recipe.