Dzmitry Bahdanau

Neural Machine Translation by Jointly Learning to Align and Translate

Introduced attention to neural machine translation: instead of squeezing the source into one vector, the decoder learns to align to and weight over all source positions — the idea the Transformer would later build its whole architecture on.