US companies invest big in AI, but measuring success remains a challenge



US companies are quickly bringing in generative AI systems and greatly raising their spending. But the biggest problem is still figuring out how much money they will get back on their investment.

Generative AI has caused a big spending surge among top US companies. A survey by Andreessen Horowitz of 70 leaders from Fortune 500 companies shows that almost all the companies asked plan to spend 2-5 times more on large language models (LLMs) and other AI systems by 2024.

In 2023, companies spent about $7 million each on base model APIs, running models themselves, and improving models. But now, large, ongoing software budgets are being used for this, not short-term innovation funds.

Measuring ROI remains a major challenge

Some companies are already paying for AI using money saved on customer service staff. One company said it cut costs for calls by 90% ($6 per call) with LLM help, so it made its AI budget eight times bigger.



Even with large investments, companies aren’t sure how to tell if AI spending is worth it. “While leaders are figuring out the answer to this question, many are taking it on faith when their employees say they’re making better use of their time,” said the analysts at Andreessen Horowitz.

Measuring the success of generative AI remains a challenge. | Picture: a16z

Even if employees can work more efficiently and with higher quality, as a study by Ethan Mollick and others suggests, the question remains whether they are actually passing on these gains in their day-to-day work. Companies often lack meaningful metrics to reliably measure success.

Productivity increases are seen as a positive indicator, reports a16z, but leaders want clearer signs like more sales, lower costs, better efficiency and accuracy – based on how AI is used.

Clear ways to track AI gains will likely stay hard for 2-3 more years. Until then, AI spending will grow even if business benefits aren’t always clear.

Multimodel AI growing

Cloud AI models are still widely used. Over half the surveyed companies use models hosted by their cloud provider.  And companies are also starting to use several AI models to improve their apps and rely less on single providers. Open-source models are becoming more popular since companies feel they offer more control and customization.


Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top