AI innovation acceleration over time

Hey there!

Welcome back to The Pulse, where we dive into interesting AI stories and trends backed by data, all presented through simple visuals.

> based on European Commission & Epoch AI’s data on AI breakthroughs

> includes notable events such as Deep Blue beating Garry Kasparov, Alphafold & GPT-2 releases, etc.

> 3 breakthroughs in 1950 to 420 in 2024

> more than 2x breakthroughs since 2022 (204); field growing at exponential rate

> avg score of top 1000 AI-ready companies: 24.5

> major difference between scores of top companies and the rest (high std deviation)

> only top 28 companies have score >50; 10 are tech & 6 are financial service companies

> RLHF doesn't necessarily improve actual performance of LLMs but misleads humans

> primarily on complex tasks where thorough human evaluation is difficult

> done by fabricating/cherry-picking evidence + writing less readable but test-passing code

> Human confidence in incorrect answers increases after RLHF; false positive rates:

• inc by 24.1% on QA tasks

• inc by 18.3% on programming tasks

> RLHF still contributes to genuine progress in LLMs, but benefits less clear for more complex tasks

.