If the emergence of generative AI has left you dismayed, know that you haven't, we haven't seen anything yet. The genie has come out of the lamp, and it will be difficult for him to get back in. On the contrary. The race is wilder than ever, and there are six projects vying to create AI models that go beyond 2 trillion parameters. Yes, you read that correctly: trillion.
The titans of AI with 2 trillion parameters
The six major projects competing for the 2 trillion metric milestone are OpenAI, anthropic, Google/Deepmind, Meta, a British government project and a still secret one. And it is a "deadly" race: economic resources are needed to keep up. Many. Between 1 and 2 billion dollars a year, to constantly update the hardware (more and more voracious of calculations and energy), hire hundreds of specialists and retain the best team members with millionaire salaries and stock options.
GPT-5: The Return of the King
After taking half the world by surprise, fired a missile at Google and having received a lot of money from Microsoft, OpenAI already has a possible KO shot in the pipeline. The one that could ensure Sam Altman's company supremacy. GPT-5 will be completed by the end of 2023 and released in early 2024, with between 2 and 5 trillion parameters.
We are unable, at the moment, to imagine its capabilities.
Claude-Next: Anthropic and its ambitious project
anthropic, the team founded by former OpenAI employees, is working on a model called Claude Next, which aims to be 10 times more powerful than current AIs. With $1 billion in funding already raised and $5 billion on the way, Anthropic expects to meet its goals over the next 18 months.
Their flagship model will require 10^25 FLOPs, using clusters of tens of thousands of GPUs. Among Anthropic's financiers is Google, which plays on multiple tables.
Gemini: Google seeks redemption with Deepmind
Google and Deepmind are collaborating to develop a competitor to GPT-4 called Gemini. The project started recently, after Bard showed di fail to compete with ChatGPT. Gemini will be a large language model with trillions of parameters, similar to GPT-4 or GPT-5, and will use tens of thousands of Google AI TPU chips for training. It is not yet known whether it will be multimodal.
Deepmind has also developed the web-based chatbot Sparrow, optimized for security and similar to ChatGPT. DeepMind researchers have found that Sparrow's quotes are helpful and accurate 78% of the time. Another top model from DeepMind is Chinchilla, trained on 1,4 trillion parameters.
The parameters of an unthinkable future
If you want to get an idea of what 2 trillion parameters means, know that the estimate of the total amount of usable text data in the world is between 4,6 trillion and 17,2 trillion parameters.
All books, scientific articles, news, the entire Wikipedia, publicly available code, and much of the rest of the Internet, filtered for quality. Digital human knowledge.
As larger models arrive, new capabilities will emerge. Over the next 6 years, there will be improvements in computing power and algorithms to scale models a thousand times, indeed much more.
Nvidia CEO predicted AI models a million times more powerful than ChatGPT within 10 years.
Can you imagine what these artificial intelligences will do with us, what they will do to the planet? Think about it quickly. We're here now.
