Some of the most powerful NLP models like BERT and GPT-2 have one thing in common: they all use the transformer architecture.