The Extreme Cost of Training AI Models

The cost of training AI models has exploded in just the past year, according to data released by the research firm Epoch AI. This development aptly shows how much more complex and capable AI models have gotten in a short time span. Last year saw the release of ChatGPT-4 in March by OpenAI, which kickstarted the global AI hype. Google followed suit with its advanced AI model, Gemini, in December.

Both systems have been much more expensive to train than previous AI models and their development has potentially cost hundreds of millions of dollars, according to the Epoch AI release. The cost of training Gemini, which is a large language model that can be inputted with text, voice commands and images, reportedly stood between $30 and $191 million even before taking staff salaries into consideration. According to Epoch AI, these can make up 29 percent to 49 percent of the final price. ChatGPT-4, the latest edition, had a technical creation cost of $41 million to $78 million, according to the source. Sam Altman, CEO of OpenAI, has in the past said that the model has cost more than $100 million, confirming the calculations.

Looking back, the cost of earlier AI models was much lower. ChatGPT-3 cost only around $2 million to $4 million make in 2020, while Gemini's precursor PaLM in 2022 took between $3 million and $12 million to train when only looking at the cost of computing. Even at these price points, keeping up with cutting-edge AI development might have proved difficult for academic or other public institutions that have traditionally been active in AI research.

While it was updated to support voice and images in fall 2023, ChatGPT-4, like its name suggests, started out based around its central text input, while Gemini and its app have been designed as a multimodal LLM from the get-go. This explains why ChatGPT's initial training cost might have been lower. On the other hand, Gemini's general focus on app delivery - for example, prompting users to snap pictures with their smartphones, pick out features in them and have them analyzed - could have warranted a higher cost.