Jacob Devlin

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Pretrained a deep bidirectional Transformer with masked-language-modeling, then fine-tuned it to a state-of-the-art on a wide range of NLP tasks. Cemented the pretrain-then-fine-tune paradigm for language.