IQ scores of reasoning and non-reasoning AI models

Hey there!

Welcome back to The Pulse, where we dive into interesting AI stories and trends backed by data, all presented through simple visuals.

> reasoning models have generally higher IQ than non-reasoning

> based on Tracking AI’s offline IQ test (questions not part of AI training)

> GPT-4.5 (research preview released last week) only model to measure up; outperforms all other non-reasoning models

> meant to have:

• improved conversations & refined personality

• better writing capabilities

• lesser hallucinations

> SWE-Lancer: OpenAI benchmark tests if LLMs can do freelance tasks (1,488 total) from Upwork worth $1M in total payouts

> Anthropic’s Claude outperforms OpenAI models

> models better at picking solutions (management) than implementing fixes (coding)

> Epoch claims AI scaling (within unsupervised learning, for models like GPT-5) can continue through 2030

> largest training runs could require 5 GW power; 170x larger than current runs

> biggest bottlenecks:

• power

• chips

• other hardware

• data (although major stride have been made in synthetic data)

.