As startups race to integrate AI into their products, they are running into a major roadblock: spiraling costs, caused by the immense computing power required to process AI queries.
CNBC reports on one startup, video game company Latitude, which found itself paying hundreds of thousands of dollars a month to process millions of AI requests from users: at least as much as its expenses for human employees.
OpenAI founder Sam Altman, creator of ChatGPT (TechCrunch/Flickr)
Via CNBC:
At its peak in 2021, Walton estimates Latitude was spending nearly $200,000 a month on OpenAI’s so-called generative AI software and Amazon Web Services in order to keep up with the millions of user queries it needed to process each day.
“We joked that we had human employees and we had AI employees, and we spent about as much on each of them,” Walton said. “We spent hundreds of thousands of dollars a month on AI and we are not a big startup, so it was a very massive cost.”
By the end of 2021, Latitude switched from using OpenAI’s GPT software to a cheaper but still capable language software offered by startup AI21 Labs, Walton said, adding that the startup also incorporated open source and free language models into its service to lower the cost. Latitude’s generative AI bills have dropped to under $100,000 a month, Walton said, and the startup charges players a monthly subscription for more advanced AI features to help reduce the cost.
The high cost of AI is a result of its computing requirements — every time an AI returns a response to a prompt, it performs billions of calculations in its mission to deliver a useful answer.
Graphics Processing Units, more commonly known as GPUs, make up the hardware that provides the computing power for most AI systems today.
However, the price of just one of Nvidia’s industry-standard GPU chips, the A100, is $10,000 — and the leading AI projects need thousands of them to power their systems.
Allum Bokhari is the senior technology correspondent at Breitbart News. He is the author of #DELETED: Big Tech’s Battle to Erase the Trump Movement and Steal The Election.