The industrial metaverse – fact or fiction?

^{Web special Fraunhofer magazine 2.2023}

So long to the 2D internet? Already, we are seeing the enormous advantages that the metaverse holds for development and production.

Frida Santos walks into her office on a Monday morning. The young engineer works for an SME that numbers among the leading companies in the solar cell market, with locations all over the world – but she herself lives in a small German town. Ms. Santos makes herself a cup of coffee, has a quick chat with a colleague and gets ready for her first meeting of the day, which she attends in a special room with no windows and partially sound-proof walls to avoid visual and acoustic distractions. The engineer puts on her VR headset and steps into a digital world. Some avatars are already sitting at a large table in the middle of the virtual meeting room – colleagues of hers, from Darmstadt, Stuttgart, Bologna and San Diego. No one had to fly anywhere or request approval for a business trip: For our fictional engineer, Frida Santos, collaborating with her colleagues is an entirely climate-neutral endeavor

“Fact or fiction?” asks Dr. Dietmar Laß. The senior research manager from the Fraunhofer ICT Group’s central office answers his own question with a definite prediction. “The metaverse is more than just hype. We can be sure that it is coming − it’s just a question of how and to what extent.” Dr. Laß and his team act as the bridging point between the activities of the various Fraunhofer institutes that are involved in the metaverse field and large companies’ increasing interest in the new digital reality. The ICT Group develops metaverse technologies and guides those companies as a reliable partner. “In recent years, we have seen significant investments by large companies, rapidly advancing technological developments and a sense of openness around this style of communication,” says Roland Busch, Chief Executive Officer of Fraunhofer’s collaboration partner Siemens. That’s why he thinks the metaverse has a big future ahead of it: “The industrial metaverse will be a space where we use the speed of software to drive innovations. Its potential for reshaping our domestic economies and industries is enormous.”

The metaverse is an important opportunity for Germany to safeguard its position as a technological powerhouse. “We have strong technology know-how in many areas, so there’s no need to hide from this new development,” points out Holger Graf of the Fraunhofer Institute for Computer Graphics Research IGD. “In the future, extended reality, software solutions for industry business processes, the virtualization and combination of 3D content, and decentralized, interoperable data platforms will have a major role to play. But what we need now is to build up an innovation ecosystem, which means we have to combine industry-based digital technologies in Germany more effectively.”

Siemens Energy is already constructing models of power plants in the metaverse, so it can predict when they will require maintenance work. The company expects that this will reduce downtime and save power plant operators 1.7 billion dollars per year. Car manufacturer BMW is planning a digital factory using the Omniverse platform by the graphic processor manufacturer NVIDIA. “The planned machines or structures will only be built in reality once the corresponding digital models have been designed in full and validated,” explains Dr. Leif Oppermann, the head of Mixed and Augmented Reality Solutions at the Fraunhofer Institute for Applied Information Technology FIT in Sankt Augustin, North Rhine-Westphalia.

Direct prototype testing with future users

The possibilities for testing are almost endless; for example, in virtual reality driving simulators, metaverse users could try out prototypes directly and then provide feedback that would immediately be taken into account in the design and manufacturing process. Similarly, the Fraunhofer Institute for Material Flow and Logistics IML has developed a simulator for DB Schenker so that their staff can practice driving forklifts. “The German rail company Deutsche Bahn uses artificial intelligence and virtual reality for digital maintenance, both for finding damage and fixing it,” relates Dr. Oppermann. When learning how to carry out repairs, mechanics can start by practicing on digital trains. “BASF is also experimenting with metaverse technologies for virtual labs, virtual showrooms and global 3D collaborations.”

The Fraunhofer Institute for Industrial Engineering IAO and the Fraunhofer Institute for Manufacturing Engineering and Automation IPA have teamed up with the Virtual Dimension Center (VDC) in Baden-Württemberg to create a large-scale metaverse project called CyberLänd. With funding from the Baden-Württemberg Ministry of Economic Affairs, Labor and Tourism, the initiative will focus on discovering what political, industrial and social potential the metaverse could offer for the south German state. First and foremost, the project team aims to help foster a deeper understanding of this complex topic. “At present, companies still don’t have any concrete prospects for development in the metaverse,” explains Prof. Katharina Hölzle, director of Fraunhofer IAO and the University of Stuttgart’s Institute of Human Factors and Technology Management IAT. “We are exploring what interactive collaboration in the metaverse might look like, how digital twins can be used on a wide scale and how you could pay people and offer on-the-job training in the metaverse.”

Next, the team is planning to identify and create a map of Baden-Württemberg companies that are already working with metaverse technologies. “We intend to conduct an empirical study to find out what expectations and requirements these companies have and then process this data so that it can be shared with the wider public,” reveals Prof. Hölzle. The project results will consist of scenarios for 2033: What role will the metaverse play in Baden-Württemberg by that point? What course of action should the government and companies take? How can participation by members of the public be facilitated? All of these results will be presented at the “Innovationsforum-Metaverse BW” (Baden-Württemberg metaverse innovation forum) event at the end of this year.

Are we at the threshold of the internet’s next great development stage?

Outside of the industry sector, the world’s attention is currently primarily focused on generative AI; however, it is most likely that in the future, the metaverse and strong AI (a form of artificial intelligence that can act independently and flexibly, with the ability to plan ahead like a human) will be used together in a complementary way. “For the metaverse to become established, we will need high-performance computers, advances in the area of self-learning systems, and scalable 3D data processing and internet bandwidth,” says Mr. Graf of Fraunhofer IGD in Darmstadt. The mathematician is conducting research on industry applications of virtual and augmented reality. However, because of the enormous amounts of computing power required to produce a good metaverse simulation, there are probably many years to go yet before industrial metaverse solutions really work in an adequate, intuitive way. “What’s more, the XR glasses market is in a state of permanent upheaval at the moment,” adds Mr. Graf. “The current headset models are not practical for everyday working use.” If major advances are to be made in terms of application in daily working situations, lighter display models that have a better field of view and function well with augmented reality applications will be needed. In fact, the ideal would be AR contact lenses that users apply directly to their eyes. But, as Mr. Graf. says, “that’s still a long way off.”

However, the experts at various Fraunhofer institutes and partner companies are already fascinated by the idea of what could happen when that possibility becomes a reality. “Experiencing situations in three dimensions and with all our senses will make it easier for us to grasp connections and expand our knowledge in the future,” says Prof. Hölzle. For Dr. Michael Bau, head of the Institute for Performance Work Health (ILAG GbR), which collaborates closely with Fraunhofer FIT at the industrial park in Troisdorf, the key to successfully implementing this technology in companies is to make sure employees and managers are involved in the process. This is why he and his team are taking into account the effects on and benefits for the people involved in the evaluation step of the project. “Geographical distance, physical disabilities and linguistic barriers could be relegated to background issues in the near future,” he predicts. “That would definitely be advantageous for all involved.”

The potential is considerable, according to metaverse expert Dr. Laß. “The metaverse can quite definitely be described as a revolution, because in the future, it will change our collective modes of living, creating, learning, working, producing, selling and consuming at a fundamental level,” he says. Prof. Vanessa Borkmann, a researcher at the Fraunhofer Institute for Industrial Engineering IAO also highlighted the metaverse’s disruptive power: “In theory, anyone can take part − they just need the right technology and access permissions. This means the metaverse is expanding our scope for action in a way that has enormous potential for the world of work.”

Now Frida Santos, our engineer from the future, is greeting her team as they all take a seat in the meeting room. Today, they plan to discuss how the technology for a solar cell prototype can be developed further. When Ms. Santos turns around, she can see a digital board where her colleague from San Diego is already presenting a list of the technical problems with the solar cell. The data is transferred over a 5G network, which is necessary for the enormous quantities of data involved; high-performance processors and memory storage are also required. Once the team has discussed solutions for the error messages, Ms. Santos goes on to the next metaverse space — her employer’s production facility in Sydney. The young engineer goes up to one of the machines and adjusts a few settings. On location in Sydney, a robot carries out her actions immediately. It’s as though Frida Santos had traveled halfway around the world within a couple of seconds.

Real objects have entered the metaverse

Yücel Uzun of Fraunhofer FIT has already started using real objects as digital twins for the metaverse. The technical manager and main developer for the mixed reality use case in the “5G Troisdorf IndustrieStadtpark” (Troisdorf 5G industrial park) project has programmed apps for displaying and using digital twins. His team is working with two industry partners, including Kuraray, a chemical company that manufactures products such as industrial polymers and synthetic microfibers using industry machines from ZWi Technologies. However, as Kuraray operates in many different countries, it very often faces short-term technical problems, such as a machine breaking down in Japan or a need to train in new employee there. Right now, the only way of resolving these issues is for an expert to travel all the way over – that means booking a flight, canceling all other meetings for days and arranging for childcare. The cost is high, in terms of both money and working days.

However, Mr. Uzun’s team has developed a solution – a system that can bring a specialist on location within seconds. With a HoloLens 2 and a VR headset, a German expert can use the metaverse to communicate with the factory worker in Japan as though they were in the same room. There is also a digital twin of the machine that needs maintenance work. “The two employees can enter the metaverse as avatars to talk and work on the virtual machine together,” explains Mr. Uzun. “In the virtual space, it’s also very easy for the German expert to send on instructions and important notes to help the factory worker.”

One of the real objects Mr. Uzun is testing is a virtual 3D model of a granulator, a machine that collects excess film and granulates it to form dust. “That sounds simple, but it’s not. We now know that in the metaverse, employees can get training on a machine or repair it directly when it’s broken.” The adjustments would then be carried out by a worker at the factory.

“We have to develop a mindset focused on ways of eliminating abuse.”

The pioneers of the metaverse have high hopes that the internet of the future will be decentralized – i.e., that it will not be controlled by any state, company or individual. This would make it possible to connect different platforms and incorporate elements from one environment in the next. But this liberality also comes with many risks. “The primary points of criticism here are focused on the uncertainties regarding jurisdiction in the international sphere, the handling of user data and intellectual property, the danger of criminal activities, the high level of resource usage and the intensification of certain web 2.0 trends,” recounts Dr. Oppermann, head of Mixed and Augmented Reality Solutions at Fraunhofer FIT. “If digital twins of factories or machines in the metaverse are used based on the same architecture as conventional internet business models, the uncertainty level will be too high for companies.” He advises getting familiar with the technology, so as to make the most of the advantages created by spacial interaction and minimize the problems. However, he adds that taking a collective approach is indispensable here. “A collective approach would enable German SMEs to benefit from excellent, versatile solutions for conducting improved remote maintenance on a digital twin of a machine, for example, without endangering their data,” Dr. Oppermann points out. “In this early stage of the metaverse, we have to develop a mindset focused on ways of eliminating abuse and implementing an ethical system, to prevent the negative elements of web 2.0 from increasing exponentially,” asserts Prof. Hölzle. Because the metaverse will only enrich our lives – rather than complicating them further – if it is built on a sound foundation of this kind.

Once these general conditions have been established, the metaverse’s potential for retail, the real estate market,tourism, education and the public sector will be enormous. For example, scientists in the Fraunhofer Institute for Industrial Engineering IAO’s research unit for Urban Systems Engineering are working to transfer existing or possible urban environments into the virtual world, using the metaverse as a platform for quickly bringing city planning and development to the prototype stage. “These digital twins allow us to simulate future scenarios, so they become more tangible for investors and users,” says Ronja Gaulinger, a research scientist in the Urban Systems Engineering research unit at Fraunhofer IAO. As well as simulating complete cities, the researchers can also create individual buildings in the virtual world, and use them to try out different layouts and look for problems. “With this system, investors can take a virtual tour inside a building before it is ever built,” enthuses Ms. Gaulinger.

Meanwhile, researchers in Fraunhofer IAO’s FutureHotel innovation network are using the metaverse to explore what hotels might look in the future (see also page 56). However, the metaverse could also be a handy tool for vacationers, for example, by allowing them to visit hotels or other travel destinations in virtual reality so that they can book the perfect vacation in real life.

“The metaverse is a turning point for companies.”

It is still hard to assess to what extent this technology will actually change our individual daily lives; however, it could be as revolutionary as the introduction of the smartphone. “Will we spend even less time in the great outdoors, with real people? Or will this technology give us the freedom to do just that?” asks Prof. Hölzle. “Maybe a counter-movement could develop here, with people making more effort to seek out the real, tangible world beyond the digital sphere, and strengthening their real-life contact with people outside the metaverse.”

One thing is certain: The metaverse will bring radical change to the German industry sector. In particular, it offers enormous advantages for companies’ innovation processes. Provided they have the right equipment, customers and suppliers from all over the world could participate in the ideation phase quickly and easily, meaning that product and service ideas can be even more accurately tailored to suit the needs of each situation. Processes like evaluating and selecting alternatives will be accelerated, because it will be possible to virtually simulate and test products such as cars or washing machines right from the planning phase. Companies could then use the results of these processes to improve products in a very simple way, as the new production data could be sent out to factories all over the world in a short space of time.

Estimates from a study by the industry consultancy firm McKinsey suggest that the metaverse could reach a value of up to 5 billion dollars by 2030. Some 95 percent of the managers surveyed in the study expect that the metaverse will have a positive effect on their sector within five to ten years, while 31 percent of them believe that it will fundamentally alter how work is done in their sector. “The metaverse is a turning point for companies when it comes to strategy,” says metaverse expert Dr. Laß. He is in no doubt: the metaverse is just too big to ignore.     

The term “metaverse” is a combination of the prefix “meta-” (meaning “beyond”) and the word “universe.” It first appeared in Neal Stephenson’s science fiction novel “Snow Crash” in 1992. In the book, the main character switches continuously between his real life in Los Angeles and a virtual world, which appears as a wide street that runs all the way around a black planet. The avatars in this metaverse constantly strive to gain more knowledge, power and money. The novel describes a dystopian world, a frightening ver­sion of reality with a very undesirable social system. A similar virtual space appears in William Gibson’s novel, “Neuromancer,” albeit under a different name: cyberspace. This concept of a digital future also fea­tures in a whole host of films, including “Ready Player One,” released in 2018.

Tech enthusiasts and companies have been develop­ing digital worlds like the one featured in “Snow Crash” since the 90s. Gaming platforms like Roblox and Second Life gave many users got a foretaste of vir­tual reality, while video games like Fortnite allow users to enter a metaverse where they can attend concerts and film premiers. After some time, tech giants started to jump on the bandwagon. Microsoft is currently developing a virtual meeting space where avatars can work together, and aims to update Microsoft Teams so that virtual and augmented reality glasses can be used with the program by the end of the year. NVIDIA, the manufacturer of the world’s fastest graphic processor, is also pitching in by developing the Omniverse, a plat­form that can be used to create metaverse applica­tions. However, at the very latest, the term’s entry into mainstream parlance can dated to the announcement that the Facebook group would be renamed Meta. The metaverse – which Mark Zuckerberg has described as “the next generation of the internet” – is set to generate 800 billion US dollars in revenue in 2030, according to the Bloomberg Intelligence finan­cial service.