Economic Turing Test
The Economic Turing Test is a concrete operational definition for the arrival of transformative AI, proposed as a replacement for the vaguer term “AGI.”
The test: if you contracted an AI agent for a month or three months on a particular job, and upon completion you learned it was a machine rather than a human, did it pass the test for that role?
The transformative AI threshold: if AI can pass the Economic Turing Test for 50% of money-weighted jobs, transformative AI has arrived.
Why this framing
Benjamin Mann prefers “transformative AI” over “AGI” because:
- AGI has accumulated too many competing definitions.
- The question that actually matters is: is AI causing economic and societal transformation?
- The Economic Turing Test is measurable: you can construct a market basket of jobs (analogous to the basket of goods used for purchasing power parity), define the 50% threshold, and evaluate.
Macro indicators
Ben also cites world GDP growth rate as an indicator. Transformative AI implies world GDP growth above ~10% per year (currently ~3%). A 3× increase in the rate of global value creation would require a qualitatively different economic regime.
Timeline
50th percentile probability of reaching this threshold: approximately 2028, per the AI 2027 superforecaster report. Ben notes this is grounded in hard empirical details (model training science, data centre scaling, compute cost trends), not extrapolation from thin air.
Where mainstream views differ
Critics of the Economic Turing Test:
- It conflates capability with economic integration — an agent could be capable enough for 50% of jobs but not yet integrated into organisations that would actually hire it (institutional stickiness).
- The 50% threshold is arbitrary; different thresholds would give very different timelines.
- Money-weighted jobs is biased toward white-collar work where AI has moved fastest; physical labour jobs may be the actual bottleneck.
Ben’s response: exact thresholds matter less than the underlying question — is AI causing massive GDP increases and societal change? The Economic Turing Test makes that question measurable rather than philosophical.