The logo for Google LLC is seen at the Google Store Chelsea in Manhattan, New York City, U.S., November 17, 2021. [Source: Reuters]
As Alphabet Inc (GOOGL.O) looks past a chatbot flub that helped erase $100 billion from its market value, another challenge is emerging from its efforts to add generative artificial intelligence to its popular Google Search: the cost.
Executives across the technology sector are talking about how to operate AI like ChatGPT while accounting for the high expense. The wildly popular chatbot from OpenAI, which can draft prose and answer search queries, has “eye-watering” computing costs of a couple or more cents per conversation, the startup’s Chief Executive Sam Altman has said on Twitter.
In an interview, Alphabet’s Chairman John Hennessy told Reuters that having an exchange with AI known as a large language model likely cost 10 times more than a standard keyword search, though fine-tuning will help reduce the expense quickly.
Even with revenue from potential chat-based search ads, the technology could chip into the bottom line of Mountain View, Calif.-based Alphabet with several billion dollars of extra costs, analysts said. Its net income was nearly $60 billion in 2022.
Morgan Stanley estimated that Google’s 3.3 trillion search queries last year cost roughly a fifth of a cent each, a number that would increase depending on how much text AI must generate. Google, for instance, could face a $6-billion hike in expenses by 2024 if ChatGPT-like AI were to handle half the queries it receives with 50-word answers, analysts projected. Google is unlikely to need a chatbot to handle navigational searches for sites like Wikipedia.
Others arrived at a similar bill in different ways. For instance, SemiAnalysis, a research and consulting firm focused on chip technology, said adding ChatGPT-style AI to search could cost Alphabet $3 billion, an amount limited by Google’s in-house chips called Tensor Processing Units, or TPUs, along with other optimizations.
What makes this form of AI pricier than conventional search is the computing power involved. Such AI depends on billions of dollars of chips, a cost that has to be spread out over their useful life of several years, analysts said. Electricity likewise adds costs and pressure to companies with carbon-footprint goals.
The process of handling AI-powered search queries is known as “inference,” in which a “neural network” loosely modelled on the human brain’s biology infers the answer to a question from prior training.
In a traditional search, by contrast, Google’s web crawlers have scanned the internet to compile an index of information. When a user types a query, Google serves up the most relevant answers stored in the index.
Alphabet’s Hennessy told Reuters, “It’s inference costs you have to drive down,” calling that “a couple-year problem at worst.”
Alphabet is facing pressure to take on the challenge despite the expense. Earlier this month, its rival Microsoft Corp (MSFT.O) held a high-profile event at its Redmond, Washington headquarters to show off plans to embed AI chat technology into its Bing search engine, with top executives taking aim at Google’s search market share of 91%, by Similarweb’s estimate.
A day later, Alphabet talked about plans to improve its search engine, but a promotional video for its AI chatbot Bard showed the system answering a question inaccurately, fomenting a stock slide that shaved $100 billion off its market value.
Microsoft later drew the scrutiny of its own when its AI reportedly made threats or professed love to test users, prompting the company to limit long chat sessions it said “provoked” unintended answers.
Microsoft’s Chief Financial Officer Amy Hood has told analysts that the upside from gaining users and advertising revenue outweighed expenses as the new Bing rolls out to millions of consumers. “That’s incremental gross margin dollars for us, even at the cost to serve that we’re discussing,” she said.
And another Google competitor, CEO of search engine You.com Richard Socher, said adding an AI chat experience as well as applications for charts, videos and other generative tech raised expenses between 30% and 50%. “Technology gets cheaper at scale and over time,” he said.
A source close to Google cautioned it’s early to pin down exactly how much chatbots might cost because efficiency and usage vary widely depending on the technology involved, and AI already powers products like search.
Still, footing the bill is one of two main reasons why search and social media giants with billions of users have not rolled out an AI chatbot overnight, said Paul Daugherty, Accenture’s chief technology officer.