Kaiming He

Deep Residual Learning for Image Recognition

Residual learning let networks go hundreds of layers deep by having layers fit a residual rather than a full mapping, solving the degradation problem. Residual connections are now standard in nearly every deep architecture, Transformers included.