2 | 23 Fraunhofer magazine AI — and what about Europe? Bigger, more powerful, more versatile: American tech giants are finding themselves in fierce competition for dominance in the world of artificial intelligence, as China is also investing billions. Europe needs to forge its own path — and it’s not too late. By Dr. Sonja Endres C hatGPT took everyone by surprise — even the AI experts. This super-powered chatbot can complete tasks that it was never trained for, and can even explain jokes. No one knows exactly how it does it. It was only trained to give responses by finding the word that was most likely to follow the pre- ceding word. ChatGPT has developed new characteristics all by itself. Experts call this “emergence” — it is a feature particular to the new, large AI models that, compared to their already powerful predecessors, are based on up to 100 times more training data. In 1997, Deep Blue defeated then-chess grandmaster Garry Kasparov; in 2011, Wat- son beat two champions of the popular U.S. game show Jeopardy; in 2016, AlphaGo defeated the world’s best Go player — these feats all garnered a great deal of attention worldwide, but they are nothing compared to the next generation of artificial intelligence. A new era began in the summer of 2020 with the release of the GPT-3 language model, which ChatGPT is also based upon, by U.S. company OpenAI. While initially only a select few people had access to GPT-3, in recent months it has become publicly available online for free in a more advanced form: ChatGPT. Dr. Gerhard Paaß, mathemati- cian and senior scientist at the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS, explains why the powerful bot keeps delivering new surprises: “These large AI models — the most famous of which at the moment is certainly GPT — can do more than just understand language, summarize texts and translate. They are trained on huge amounts of general data and can be easily adapted to a wide variety of tasks.” Experts refer to this phenomenon as homogenization. Task-specific training is no longer necessary; the models respond to instructions and provide the basis for a wide range of applications, which is why they are also called foundation models. ChatGPT can program like a professional developer, collate specific information from medical texts and pass law exams. Foundation models are also multimodal, which means they can process various media, including speech, images and videos, in addition to text. In order to quickly drive development, Microsoft has invested additional billions of dollars in OpenAI; as a result, the company has since followed up with an even larger model, GPT-4. The technical details have not yet been published, but a variety of test results have suggested that it can reproduce facts even more reliably than GPT-3. Meta and Google are also working hard to create their own models. Some 73 percent of the major AI models are currently being developed in the U.S., and 15 percent in China — but as it stands, less than 2 percent in Europe, including models such as Luminous by the German startup Aleph Alpha. The huge impact of language on AI This will soon change, according to the authors of the LEAM feasibility study, which was presented to the pub- lic in January. Conducted by Fraunhofer IAIS together with other research institutions, trade associations and companies, the study describes the conditions that must be put in place if Germany is to maintain a competitive position when in comes to developing foundational and language models for AI. The study was funded by the German Federal Ministry for Economic Affairs and Energy. “Training AI with European languages is a matter of immense importance,” says Dr. Paaß, who took part in the study. Language has a huge impact on AI, as language represents culture, with all its unique characteristics, norms and values. “Only 3 percent of the training data for GPT-3 was in German. This means, for example, that detailed facts about German history, geography and technology, as well as our norms surrounding data back to page 1 53 . 3 1 – 8 . p p , i 0 2 0 2 / 3 k n h c e T & r u t a N : n i , i ? n e k n e d n e n h c s a M n e n n ö K , f l o d u R , g n i s i e S d n u k n a r F , n n a m t t i D : I K r e d e t h c i h c s e G : e c r u o S s e g a m p d d i , s s e r p n o i t c a , s e v i h c r A y t i s r e v i n U n o l l i e M e g e n r a C e h t f o y s e t r u o C , y e n r u o d M i j I K : o t o h P