Concept
Sequence-to-Sequence Learning
Mapping one variable-length sequence to another with an encoder–decoder, the framing behind neural translation and beyond.
Papers
-
Sequence to Sequence Learning with Neural Networks
Framed translation as sequence-to-sequence learning: an LSTM encoder compresses the source into a vector that an LSTM decoder unrolls into the target. The encoder–decoder skeleton everything that follows hangs off of.
arxiv.org ↗ -
Neural Machine Translation by Jointly Learning to Align and Translate
Introduced attention to neural machine translation: instead of squeezing the source into one vector, the decoder learns to align to and weight over all source positions — the idea the Transformer would later build its whole architecture on.
arxiv.org ↗