Photo by Georg Arthur Pflueger on Unsplash

“We tested Bort on 23 NLU tasks, and on 20 of them, it improved on BERT’s performance — by 31% in one case — even though it’s 16% the size and about 20 times as fast.” — Adrian de Wynter

The objective of the present article is to show how to overcome the few issues you’ll encounter when you try to get Bort running in the popular and convenient environment of Simple Transformers. This environment is built on top of Hugging Face Transformers.


First you create something that works well, then you optimise it.

Bort optimises BERT. Note that I…

Cees Roele

Language Engineer, Python programmer, Scrum Master, Writer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store