Disinformation und cybersecurity

“Hardly anyone would have thought the methods of making deepfakes would evolve at such a breakneck pace,” says Prof. Martin Steinebach. “Two years ago, you still needed good actors to make deepfake videos seem even remotely realistic, but today, some deepfakes almost look better than the original.” Steinebach, a multimedia forensics expert, has been working on technical recognition methods at the Fraunhofer Institute for Secure Information Technology SIT in Darmstadt for years now. He studies how different methods of creating the fakes work, what traces they  leave, and how those can be detected automatically.   

What is new is not only the quality, but also the quantity, of disinformation campaigns: Spamouflage, a campaign operated by a Chinese network, spans hundreds of thousands of accounts on more than 50 websites, including all the major social networks. These accounts generate millions upon millions of posts. Upon until 2022, the spammers focused on spreading pro-Chinese narratives, but now they have turned their attention to the U.S. elections and are spreading targeted disinformation. The really treacherous thing about it is that where the Spamouflage posts used to be readily identifiable as fakes due to glaring spelling and grammar errors, which limited their impact, AI has been a game changer.

Targeting the private sector

An example from Hong Kong shows just how perfectly fake worlds can work. An employee of an international corporate group there sat through a whole videoconference with multiple employees of his company – but it was all fake. His alleged colleagues prompted him to transfer almost 24 million euros to the people behind the scheme. They had evidently hacked into the company’s internal video records beforehand and used AI to generate voices to go with the videos. The target of the fraud did not notice that he was the only real person in the conference until afterward, when he talked to his real boss on the phone.

Scenarios like this one could become more commonplace in Germany in the future as well. The German Federal Criminal Police Office published a report on the national cybercrime situation in 2023 this past May. The report shows that the number of crimes committed from other countries against victims in Germany rose 28 percent from the previous year. The attacks caused 206 billion euros’ worth of damage.

Disinformation is the world’s biggest issue today

“The most significant challenge of our time is not climate change, the loss of biodiversity or pandemics. The biggest issue is our collective inability to distinguish between fact and fiction,” concludes the Club of Rome – the well-known organization that, over 50 years ago, denounced the environmental impact of economic growth. The consequences of digital manipulation are loss of trust and a polarized, uncertain society where people can no longer agree on fundamental facts. Where trust erodes, people tend to believe information that confirms their own viewpoints. And that further shrinks the factual basis available for democratic discourse. With all these factors in play, it is little wonder that 81 percent of people in Germany think disinformation is a danger to democracy and social cohesion, as this year’s study titled “Verunsicherte Öffentlichkeit” (Unsettled Public) from the Bertelsmann Stiftung shows.

Sixty-seven percent of respondents believe disinformation campaigns originate with groups of protesters and activists, while 60 percent blame bloggers and influencers and nearly half say foreign governments are at fault. The study also permits a comparison with the United States, where uncertainty about and perceptions of disinformation are even more pronounced than in Germany. While 39 percent of Americans surveyed worry about being deceived by disinformation themselves, so they are increasingly verifying content with a more critical eye, Germans firmly believe in their own judgment, with only 16 percent saying the risk that they might be influenced by disinformation themselves is high or very high and 78 percent saying it is low. This self-confidence could prove to be misplaced now, with the new possibilities created by AI.

Where AI can help to identify deepfakes

There are a wide range of methods used for manipulation. In face swapping, two faces from different photographs are simply swapped. In facial reenactment, a person records a fake text and remotely controls a target person in a real video with their movements and gestures  – in real time. “What is new is the ability to use voice cloning and lip synchronization to create convincing videos of any kind of content,” says Steinebach, the forensics expert from Fraunhofer SIT. “Existing videos are simply given an AI-generated fake soundtrack, and the lip movements are synchronized to match.”

To recognize images or videos that have been manipulated in this way, Steinebach and his team rely on a combination of deep learning and traditional signal processing, which can help to identify the blurry or slightly washed-out structures in an image that are typical of deepfakes. “We measure the frequencies of certain sections of the image and compare them to other parts, such as the background. If there are any discrepancies, it could be a sign of a fake.” Since deepfakes only replace one area of a real video, that area also has different statistical properties, which can be identified using a more advanced pixel analysis. If only the soundtrack is changed, not the video itself, recordings that are already known are often easy to find using a reverse image search. One telltale sign is if the gestures are the same, but the movements of the lips differ.

Automatic detectors: caution warranted

There are already a number of AI-based detection tools on the market that promise simple, cheap technical solutions. But how realistic is this idea of reality at the push of a button? Steinebach warns against relying on online detectors in videoconferences or browsers. “The error rates are still too high for that, and it would cause more uncertainty than benefit if there were constant warnings.” Instead, he argues that these kinds of solutions should be used only in conjunction with the eyes of multiple trained experts. Forensic reports that incorporate additional factors, such as sources, falsification scenarios, plausibility checks, and potential circumvention strategies, offer significantly greater certainty. But owing to the cost, these kinds of analyses are typically commissioned only in areas that are especially critical from a security standpoint or of special legal relevance.

A recent example shows how much caution there should be given that the standard detectors currently available could even be abused, with error rates generally still in the double digits: After the Hamas attack on Israel on October 7, 2023, the Israeli government published multiple photos of the burned corpses of babies as proof of the gruesome acts committed by the terrorists. One of the photos was erroneously flagged as fake. The terrorists then used AI to generate a new picture in which they swapped the children for dead dogs and said that was the real photo. This was intended to discredit all the other photos that were not flagged as fake by association, suggesting that the Israelis were deceiving the public.

The British royal family also had its own experience with this kind of “false positive” this past spring. All it took to touch off a firestorm of scandal was for an online detector to find irregularities in a family photo of the royals. A whole host of experts chimed in that there was nothing suspicious about it, just standard photo editing like the kind many amateur photographers do, but their voices were almost drowned out in the breathless reporting.

With detection of fakes still that error-prone, a different method can help to at least recognize the originals: “The only strategy people can be relatively sure of today is a positive signature: Newer digital cameras leave a cryptographic signature in their pictures that is difficult to falsify. There are also cell phone apps that do this,” Steinebach says. “Big tech companies are also working on new security strategies in which, like with blockchain, all of the steps in processing an image are signed and stored, for example.” The royals and others should be pleased to hear that.

Deepfake audio: Don’t believe your ears

“Voice cloning” techniques are increasingly popular with criminals. Attackers used voice cloning to influence the Slovakian parliamentary elections in 2023. Two days before the election, social media was abuzz with an audio clip purporting to be of a prominent journalist and the head of the Progressive Slovakia party, talking about how to rig the elections. The fake discussion never happened, but the clip reached thousands of users. Democratic voters in the U.S. got a surprise at the start of this year, too: a personal call from Joe Biden asking them not to vote in the primaries. The U.S. president’s voice was AI-generated, and the calls were made automatically.

“Just 20 seconds of audio material is enough these days to filter out typical voice characteristics and use them to generate whole new sentences,” says security expert Nicolas Müller from the Fraunhofer Institute for Applied and Integrated Security AISEC in Garching, describing a machine learning technique known as few-shot learning. A short clip of a public speech is sufficient to manipulate discussions or presentations. A year ago, many programs still could not manage to consistently falsify a minimum of 16,000 data points per second, but with AI, that’s not an issue now. Plus, it works in near real time. Still, Müller says there are clues to be found in these cases, too, if you really listen closely: “An AI-generated voice is monotonous in some cases, with unnatural intonation, subtile delays, and little emotion.”

The security experts at Fraunhofer AISEC also use AI for counterattacks: They train and develop solutions to automatically unmask audio deepfakes. To do this, they first generate artificial audio and video data and then use the information to develop algorithms that recognize the fakes from the tiniest discrepancies, barely noticeable to humans. The Fraunhofer AISEC researchers provide their developments to the general public online at deepfake-total.com. Links or files can be uploaded there to test whether the content is fake. “These kinds of tools generally have an easier time recognizing recordings from known generators than content from entirely new sources,” Müller says.

The increasing quality of audio fakes is also increasing the direct risk of fraud. Müller advises anyone who receives a suspicious call from a relative or friend asking them to transfer funds quickly or disclose confidential information to hang up and call back, if at all possible using a different channel. He and his team are also researching methods to make systems for facial recognition or voice authentication, such as voice ID systems, more robust and resilient to manipulation and attacks. Aside from the risks, the researcher also believes it is important to point out the possibilities unlocked by speech generators. “For example, they can help people with speech impediments to be understood better, or even at all, by voice assistants or other people.” Right now, for example, Google is researching AI that can translate atypical, hard-to-understand speech into fluent speech.

Speed versus careful attention: the social media dilemma

Where fake content is concerned, it is also worth looking at how it spreads. Social media platforms are obligated to flag suspicious content, but they have a hard time keeping up. After all, it’s a lengthy process. First, they need to recognize whether the content is problematic. This is either done by internal research teams, or the company relies on user reports, which is anything but reliable and effective. That’s because nothing is black and white: “Telling whether content is entertainment or actual news, whether it should be taken seriously or is exaggerated satire, is hard for both people and machines,” Steinebach says. “And something else people often overlook is that a lot of disinformation isn’t just flat-out lies. It’s truth, spread maliciously or taken out of context.” The next step is to decide whether a post is relevant in the first place, meaning that it is frequently shared or intended to have a destabilizing effect. Only then can the content be marked accordingly, deleted, or blocked. “But by then, days have gone by, and the fake news has been spread all over the world,” the forensic expert says. So speed is of the essence. And therein lies the rub: Responding swiftly can hamper the careful attention needed to conduct a thorough review.

Even when fake content has been removed from the big platforms, it often circulates for a long time afterward on messenger services like Telegram. The researchers at Fraunhofer SIT are scrutinizing these channels as part of the Dynamo project. To understand the dynamics of disinformation campaigns there, they are studying how the content spreads via messengers and how they interact with other channels as well. They are interested in which properties make these services prone to disinformation campaigns, whether there are patterns to the spread of fake news, and what counter-strategies can be identified as a result. The goal is to develop technical aids and approaches to combat fake news and provide them to the public, which will likely take place this fall.