OpenAI CFO Sarah Friar has proposed a new metric called 'useful-intelligence-per-dollar' to measure AI investment returns, arguing that traditional software metrics fail to capture AI's value. This comes as companies face increasing pressure to demonstrate measurable business value from their AI investments.
TL;DR
- OpenAI CFO Sarah Friar introduces a new metric to measure AI investment returns based on work accomplished.
- Traditional software metrics are insufficient for capturing AI's business value, according to Friar.
- Companies are under pressure to show measurable returns from AI investments, with only 12% of CEOs reporting both cost and revenue benefits.
What happened
OpenAI CFO Sarah Friar has proposed a new metric called 'useful-intelligence-per-dollar' to measure AI investment returns. In a blog post, Friar argued that traditional software metrics, such as seats, active users, and renewals, fail to capture the business value that AI can create. Instead, she suggested that companies should focus on metrics like the number of tasks completed successfully with AI, the total cost of completing those tasks, and the cost per successful task over time.
Friar's proposal comes as companies face mounting pressure to show that AI investments are delivering measurable business value. Worldwide spending on AI is forecast to total $2.59 trillion in 2026, a 47% increase year-over-year, according to Gartner. However, a study released by PwC in January found that only 12% of CEOs said AI had delivered both cost and revenue benefits. Overall, 33% of respondents reported gains in either cost or revenue, while 56% said they had so far seen no significant financial benefit.
Why it matters
The push for new metrics comes amid growing scrutiny of AI token consumption, or the amount of AI processing companies pay for when using large language models. Palantir CEO Alex Karp recently argued that many business leaders are growing increasingly frustrated with the economics of LLMs, questioning whether rising token costs are translating into ROI. 'The enterprises are just tired of it,' Karp said in a CNBC interview.
Friar's proposal is significant because it offers a new way to think about AI investment returns. By focusing on specific workflows and outcomes, such as resolving customer issues, completing software changes, reviewing contracts, or preparing forecasts, companies can better understand the value that AI is creating. This approach also takes into account the cost of AI, including model and computing expenses, as well as factors such as employee time, human review, and rework required to achieve a usable result.
Key facts
- OpenAI CFO Sarah Friar proposed a new metric called 'useful-intelligence-per-dollar' to measure AI investment returns.
- Traditional software metrics, such as seats, active users, and renewals, fail to capture the business value that AI can create, according to Friar.
- Worldwide spending on AI is forecast to total $2.59 trillion in 2026, a 47% increase year-over-year, according to Gartner.
- Only 12% of CEOs said AI had delivered both cost and revenue benefits, according to a PwC study released in January.
- Palantir CEO Alex Karp argued that many business leaders are growing increasingly frustrated with the economics of LLMs.
- Friar's proposal focuses on metrics like the number of tasks completed successfully with AI, the total cost of completing those tasks, and the cost per successful task over time.
- The cost of AI should include not only model and computing expenses but also factors such as employee time, human review, and rework required to achieve a usable result, according to Friar.
Context
The AI industry is currently facing a challenge in demonstrating the business value of AI investments. While spending on AI is increasing, many companies are struggling to show measurable returns. This has led to growing scrutiny of AI token consumption and the economics of large language models.
OpenAI's proposal for a new metric to measure AI investment returns is a significant development in this context. By focusing on specific workflows and outcomes, companies can better understand the value that AI is creating. This approach also takes into account the cost of AI, including model and computing expenses, as well as factors such as employee time, human review, and rework required to achieve a usable result.
The broader AI landscape is evolving rapidly, with new technologies and applications emerging all the time. As companies continue to invest in AI, it will be increasingly important to have metrics that can accurately capture the business value that AI is creating.
