Is GenAI Delivering?

Having made its debut at the end of 2022 in the form of ChatGPT 4 and other large language models, generative AI stormed through 2023, fueling enormous expectations and driving the kind of heady investiment that hadn’t been seen since the dot.com bubble of the early 2000s.

With so much enthusiasm riding on the new technology, the big question was whether or not GenAI would deliver the kind of concrete results that would justify – or not – the massive funding.

Great expectations

At the beginning of the year commentators agreed that the time had come for the technology to show its stuff:

“2024 is going to be a year of reckoning, ” according to the Verge article Artificial Investment.

“2024 is the year to turn GenAI’s magic into business impact,” predicted Boston Consulting Group in the article From Potential to Profit with GenAI.

The message was clear – the enthusiasm of 2023 was about to undergo a reality check in 2024.

The glass half empty?

In fact, there were already signs of scepticism at the beginning of the year. The Verge pointed out that shares in leading AI players Microsoft and Alphabet had fallen by 2% and 7% respectively in spite of reporting strong revenue growth for the last quarter of 2023.

Expectations were to blame: “Investors weren’t expecting good. They were expecting perfect.”

Investor caution was probably also fuelled by memories of other recent tech ‘revolutions’ that had failed to live up to their promoters’ promises - in particular blockchain and the metaverse.

Another factor making investors jittery in January was the sheer cost of the technology: “AI is expensive. Take OpenAI, for instance; in December 2023 … OpenAI made roughly $167 million that month. It is nonetheless operating at a loss and will likely need to raise “tens of billions more” to keep going, the Financial Times reported.”

Even at this early stage, doubts were beginning to be raised about whether GenAI applications would be able to perform at the level expected.

The Verge notes: “Even OpenAI is trying to backpedal on the hype. In December, OpenAI chief operating officer Brad Lightcap told CNBC that he keeps having to explain to people that AI can’t dramatically cut costs or bring back growth for struggling companies.”

“Morgan Stanley’s AI chatbot is being bypassed by wealth managers because people want to talk with other people, The Information reported. News operations attempting to replace journalists with AI-written articles have faced backlash as those articles have been wrong, offensive, or useless.”

The glass half full

But The Verge’s assessment of the potential for GenAI was not entirely negative. One sector it saw as obtaining concrete and immediate benefits was that of IT itself: “The first place close observers like Greenstein believe adoption may occur widely is IT — especially around software development, customer service, and back-office tasks, such as processing inbound forms. That makes Microsoft’s Copilot, the “AI companion,” an indicator for the overall field.”

Other more sunny forecasts for GenAI application are to be found in the Boston Consulting Group’s January report cited above. Based on a wide survey of 1400 company executives, the BCG report revealed high expectations of the new technology while noting that only a minority – around 10% of those polled - were taking the necessary steps to realize the gains that properly deployed GenAI can deliver.

“While almost all executives now rank AI and GenAI as a top-three tech priority for 2024, 66% of leaders are ambivalent or dissatisfied with their progress on AI and GenAI—and only 6% have begun upskilling in a meaningful way.”

BCG found that while some companies were leveraging GenAI effectively to obtain higher efficiency and cut costs, most did not yet have the necessary training or governance in place to make safe, effective use of the new technology.

Many were adopting a ‘wait and see’ approach: “…two-thirds of the executives we surveyed believe that it will take at least two years for AI and GenAI to move beyond the hype, and 71% are focused on pursuing limited experimentation and small-scale pilots. Some 90% of leaders fall into one of these two categories.”

But there was little room for doubt in the Boston group’s bullish assessment: GenAI would deliver for companies that made the right investments. Failure to obtain the potential benefits would reflect the limitations of management – not the technology.

Where does GenAI stand on the Gartner ‘hype cycle’?

Fast forward to the middle of 2024 and one of the first indications of how the year was going for the new technology was the Gartner ‘hype cycle’.

The Gartner cycle is based on the idea that technologies pass through three stages on their way to becoming established: the Peak of Inflated Expectations, The Trough of Disillusionment, and the Slope of Enlightenment.

So where is GenAI on this cycle?

The report, which came out in the middle of June, placed it just past the Peak, beginning its descent into the Trough of Disillusionment.

Source: Gartner - Hype Cycle for Artificial Intelligence, 2024

It also estimated the time it would take to reach the Plateau of Productivity at two to five years – rather longer than the boosterish forecasts being made at the beginning of 2024 and broadly in line with the more pessimistic sources cited in the Verge article.

Gartner's overall view: “Investment in AI has reached a new high with a focus on generative AI, which, in most cases, has yet to deliver its anticipated business value. “

The missing $600 billion

Just how much would that business value have to be to justify the level of investment? At the end of 2023 David Cahn, COO of Sequoia Capital, had calculated the total ongoing investment in generative AI by the simple method of taking Nvidia revenues from AI chips and adding the necessary margins. It came out to $200 billion.

Cahn repeated the exercise in June 2024, by which time the figure had ballooned to $600 billion - the revenue that Gen AI would have to generate to allow its investors to break even. While Cahn remained optimistic about the future prospects for the technology, he was clear that there was not going to be an immediate return on such a large investment: "... we need to make sure not to believe in the delusion that has now spread from Silicon Valley to the rest of the country, and indeed the world. That delusion says that we’re all going to get rich quick, because AGI is coming tomorrow..."

A cold shower from Goldman Sachs

A week after the publication of the Gartner report, investment bankers Goldman Sachs issued a report on the technology and its prospects which made big waves in the investment community.

Entitled GenAI: Too Much spend, too little Benefit? , it consisted of a collection of interviews and discussions with industry experts, many of whom were highly sceptical of the ability of GenAI technology to repay the kind of investments which had been made and were continuing.

Perhaps the most negative viewpoint quoted in the report is that of MIT institute Professor Daron Acemoglu who stated: “Given the focus and architecture of generative AI technology today... truly transformative changes won’t happen quickly and few—if any—will likely occur within the next 10 years.”

Acemoglu also saw the range of applications as being constrained by GenAI's intrinsic limitations: “Many tasks that humans currently perform, for example in the areas of transportation, manufacturing, mining, etc., are multifaceted and require real-world interaction, which AI won’t be able to materially improve anytime soon.”

Goldman’s Jim Covello pointed out that the GenAI substitution model makes little economic sense: “Replacing low-wage jobs with tremendously costly technology is basically the polar opposite of the prior technology transitions I’ve witnessed …”

He also expressed doubts about the transformative power of the technology: “Many people seem to believe that AI will be the most important technological invention of their lifetime, but I don’t agree, given the extent to which the internet, cell phones, and laptops have fundamentally transformed our daily lives, enabling us to do things never before possible, like make calls, compute and shop from anywhere.”

“Currently, AI has shown the most promise in making existing processes—like coding —more efficient.”

Nor was Clovello confident that transformative applications would emerge going forward:

“AI bulls seem to just trust that use cases will proliferate as the technology evolves. But eighteen months after the introduction of generative AI to the world, not one truly transformative—let alone cost-effective—application has been found.”

Perhaps taken aback by the reception of its July report, in September Goldman Sachs published a note, Why AI stocks aren’t in a bubble , in which it offered a more nuanced view, distinguishing between different kinds of players in the AI segment and concluding that valuations, while high, did not constitute a ‘bubble’.

IBM looks more on the bright side

Shortly after the publication of the Goldman Sachs report, IT giant IBM issued its own view of the status of GenAI technology: Will generative AI live up to its hype? 

While noting the critical points of the Goldman Sachs article, the IBM piece points out that investment in GenAI was still continuing apace: “According to KPMG, a full 20% of global VC funding was captured by AI companies during the second trimester of 2024”.

But tech entrepreneur Gilles Raymond observed that much of the revenue growth generated by the technology was not being enjoyed by AI practitioners: “It is … interesting to see that the real winners of AI are those who don’t do AI but sell servers, microchips, etc. They truly are the big winners in this race.”

The tech sector is the clearest winner

The IBM report also confirmed one of the trends visible at the beginning of the year: it’s in the tech sector itself that immediate gains from GenAI adoption are being reported, while in other sectors the advantages are not nearly so clear.

They quote Cyril Maury, partner at international consulting firm Stripe Partners: “We do have two types of clients: those who are in tech and want to develop AI and integrate it in their workflows. Others are ‘legacy’ businesses, that fear disruption. They see AI as a very powerful tool, but they don’t see the benefits.”

The takeaways

As we approach the fourth quarter of 2024, experimentation and investment in GenAI continues - but it’s already clear that this will not be the ‘year of reckoning’ foreseen at the beginning of the year.

Nevertheless, some conclusions can be drawn.

• The big winners so far are the suppliers of chips and infrastructure for AI deployment. Investments in these areas continue to be very lucrative.
• The tech sector still presents the clearest examples of AI-powered ROI. Goldman Sachs analysts note: “ In the code development domain, AI has automated low-level code writing, freeing up developers to work on more complex and productive tasks.” (However, a recent survey by Uplevel Data Labs of 800 developers, failed to find any productivity gains from using GenAI coding tools and revealed a 41% increase in the incidence of bugs in the software produced.)
  
• In most other sectors GenAI investments have not yet delivered the value that would justify the large-scale investment in the technology. It's not always clear if this is due to slowness of uptake by potential users or limitations of the technology itself.
• It’s probably still too soon to say if GenAI will deliver the kind of radical business transformation promised by its most enthusiastic promoters. It certainly won’t happen this year and Gartner’s more sober estimate of “two to five years” seems nearer the mark.

Early examples of AI delivery

In the meantime, some sectors have seen specialized applications of GenAI technology that are already achieving significant gains in productivity and efficiency.

In the news-media sector, for example, dedicated GenAI models are adding value to publishing operations at several points in the news creation cycle, from authoring assistance to automated print page layout and subscription management.

Find out more about how generative AI technology is enhancing productivity in the news-media sector.