From recurrent networks to GPT-4: Measuring algorithmic progress in language models

In 2012, the best language models were small recurrent networks that struggled to form coherent sentences. Fast forward to today, and large language models like GPT-4 outperform most students on the SAT. How has this rapid progress been possible?

from Tech Xplore - electronic gadgets, technology advances and research news https://ift.tt/1BDe9mQ