On March 6, Business Insider published an article saying that although Putin hopes that Russia will stand out in the global AI competition and surpass China and the United States, in fact, Russia’s flagship large language model is not outstanding, and can even be said to be very backward. In December 2024, Putin said at the "Journey to Artificial Intelligence" international conference,Russia not only wants to catch up with China and the United States in the field of artificial intelligence, but also wants to become a global leader.
Now, on the Russian version of the Large Language Model Competition Platform (LLMArena),Russian large model GigaChatMAX tied for eighth place, lagging behind multiple versions of Claude, DeepSeek and ChatGPT. Another big language model in RussiaYandexGPT4Pro ranks even lower, tied for 18th.
In the English version, both models areDid not enter the ranking of more than 170 large language models.
GigaChatMAX is developed by Russian state-owned bank Sberbank. When the latest version was launched last November, its Moscow-based lead developer Evgeny Kosarev said it was "close to GPT4o in Russian and English quality" but experts told Business Insider,GigaChatMAX is months behind its U.S. and Chinese competitors.
Lukash Oleinik, a senior researcher at King's College London, said that at present, GigaChatMAX "performs mediocrely." The models scored "much lower" on benchmarks used to test the effectiveness of artificial intelligence. He also added,They don't surpass any cutting-edge models, nor are they anything particularly innovative.
Ben Dubo, senior fellow at the European Center for Policy Analysis and chief technology officer of data analytics company Omelas, added that GigaChatMAX lacks advantages in many aspects. Dubo wrote in the Moscow Times in January that while it performed well in math, its Russian language capabilities lagged far behind most leading Western and Chinese large language models in some benchmarks.
He said that the American large language model is one year ahead of the current level of GigaChatMAX in the industry standard "large-scale multi-task language understanding" (MMLU) test. Dubbo also told Business Insider that most AI is being tested on more advanced benchmarks and that MMLU is "almost considered obsolete at this point."
Dubbo said:"Outperforming U.S. and Chinese models in Russian language prompts is a top priority of the Russian government's artificial intelligence strategy, but MAX has not yet achieved this goal."
Samuel Bendett, a Russian military technology expert at the Center for Strategic and International Studies, told Business Insider that for Russia, artificial intelligence is a "status symbol." But according to the global artificial intelligence rankings produced by British media start-up Turtle Media, among the five major countries: the United States, China, France, Britain and Russia,Russia is the only country not at the top of the list, ranking 31st.
Bendett said that Russia “doesn’t have companies like OpenAI, Microsoft, Google, Huawei or Alibaba.”
In February, as DeepSeek stirred up the global AI landscape, Russian state-owned bank Sberbank planned to launch a joint AI project with Chinese researchers. Alexander Wei Jiaxin, the bank's first deputy chief executive, told Reuters that DeepSeek's success fits with Sberbank's strategy of finding low-cost solutions rather than high-investment "American solutions."
Wei Jiaxin said: "Sberbank has many scientific researchers. We plan to use them to carry out joint research projects with Chinese researchers."