Transformers are becoming the most impactful tech of the decade, but it's important to ensure they are used in an ethical and transparent manner.
- Transformers have enabled huge advances in NLP like next sentence prediction, question answering, and text summarization.
- Transformers have become the new default way of building technology.
- There are important limitations today, such as bias in models like BERT.
- It's important to invest heavily in machine learning ethics and transparency.
- Initiatives like model cards and data measurement tools can help ensure AI fairness.
BERT is extremely biased for men, predicting lawyer, carpenter, doctor, weatherman, and mechanic, while for women it predicts nurse, waitress, teacher, maid, and prostitute. This highlights the problem of bias with these models.