Are Transformers Becoming the Most Impactful Tech of the Decade?

Transformers are becoming the most impactful tech of the decade, but it's important to ensure they are used in an ethical and transparent manner.

• Transformers have enabled huge advances in NLP like next sentence prediction, question answering, and text summarization.
• Transformers have become the new default way of building technology.
• There are important limitations today, such as bias in models like BERT.
• It's important to invest heavily in machine learning ethics and transparency.
• Initiatives like model cards and data measurement tools can help ensure AI fairness.

BERT is extremely biased for men, predicting lawyer, carpenter, doctor, weatherman, and mechanic, while for women it predicts nurse, waitress, teacher, maid, and prostitute. This highlights the problem of bias with these models.

Clement Delangue, CEO of Hugging Face insightfully describes through the evolution of Transform models. These foundational models have generated unprecedented excitement in academia and industry, enabling huge advances in NLP like next sentence prediction, question answering, and text summarization. He shares why it's important to ensure they are used in an ethical and transparent manner. With that in mind, he discusses the work being done to ensure AI fairness in foundational models.