Researchers at the Fraunhofer-Gesellschaft have developed an AI-driven technology that allows the perimeter advertisements shown during live broadcasts of soccer games or other events to be changed without viewers noticing. This means every TV station can show its own content on the boards. The Swiss company ViboTec AG is bringing this technology to the market.
For sports fans who regularly tune in to track and field sports, soccer, or other events on their TVs, the long advertising boards on the sidelines are a familiar sight. These earn good money for sports event organizers and media rights distributors, and allow advertisers to reach an audience of millions. However, in the age of digitalized advertising with precise audience targeting, this format has its drawbacks: All viewers see the same advertisements, regardless of which country or region they are in.
Now, the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS has developed a technology that makes it possible to adjust the content on the perimeter advertising boards to suit each target group. Then TV stations will be able to use the boards to show content that advertisers have tailored to a certain audience or content focused on a specific country. When a soccer game is broadcast worldwide, millions of viewers will see the same perimeter advertising boards, but each with different content.
Fraunhofer researchers from Sankt Augustin near Bonn have combined the chroma key process, an established feature in TV productions, with an innovative technology they developed. In chroma keying, also known as green screen or blue screen, TV presenters stand in front of a monochrome colored surface, which you can overlay with any image or video. It then appears to the viewer as though the anchorperson were standing in front of a winter landscape, for example. The Fraunhofer team uses magenta as the background color; however, almost any color can be used if the colors of the players’ jerseys don’t stand out enough against a magenta board.
But how exactly does this work for perimeter advertisements? Modern perimeter advertisements use LED panels. These are controlled electronically and present a stream of changing content.
This is how Ulrich Nütten, head of the Media Engineering department, explains the procedure developed at the institute: “The streams on the LED boards consist of a sequence of individual images, each of which is displayed for 20 milliseconds. We shorten the display time to 18 milliseconds, and the board displays a solid magenta screen for the remaining two milliseconds. This flash of magenta is too quick to be perceived by the human eye. The TV camera’s recording timing and duration are set so that it only records these flashes. That means the camera doesn’t see the real advertisements, only the boards with the solid color. This is where chroma keying comes in. The magenta area in the image sent to the TV studio by the camera at the stadium is overlaid with new content. This creates an advertising stream that seamlessly blends into the TV image.” While spectators in stadiums see the actual perimeter advertisements, people watching on their TV sets are shown different advertisements. However, for them, it's as if the advertisements are displayed on the perimeter boards in the stadium. It doesn’t even pose a problem if one of the players stands in front of a board, or runs back and forth in front of it. “The technology is now so advanced that there's no fringing, color distortion, or blurriness in those situations,” says Nütten.
This was yet another problem that Fraunhofer researchers had to solve. When a camera tilts or pans, the perspective of the perimeter advertisements looks distorted. The externally generated advertising content has to be adjusted so that it always stays within the confines of the boards. The IAIS team solved this problem by placing tracking modules on the cameras. These modules continuously record the angle of the camera and panning movement. AI-driven software uses this data to constantly recalculate the angle at which the boards are shot and to compensate for the resulting distortions in perspective. Despite the considerable computing capacity that requires, the system has no need for a mainframe computer. “Two commercial-grade PCs with powerful graphics cards are enough. One PC detects the magenta boards on the TV image and the other overlays them with the ad content.”
For this project, Fraunhofer IAIS is focused on advancing the development of the basic idea, implementing the idea on a technical level, and designing the corresponding software in a user-friendly way. However, when it comes to marketing this innovative technology, Swiss company ViboTec AG have stepped in. ViboTec CEO Marc Pfister says: “With this virtual perimeter advertising technology, we’re not just serving an existing market, we're actually creating a new market for live TV broadcast advertising with precise audience targeting. In future, this will allow for entirely new business models, such as personalized advertising on streaming services.”
Researchers at Fraunhofer IAIS have already started working on the next step: supporting TV broadcasts in ultra-high-resolution 8K format.
Fraunhofer-Gesellschaft