Anyone who has visited one of the big online video portals or TV broadcasters’ media libraries to search for a video clip is already familiar with the search engines tasked with seeking out and flagging video footage. However, these engines have their weaknesses. Their results are based on automatic search algorithms that often go by text-based information alone. Although they can be used to locate and identify videos, a comparison of individual sequences is still very difficult. To make search engines even smarter, the Fraunhofer Institute for Digital Media Technology IDMT in Ilmenau has developed a piece of software called “NewsHistory” that will now make full use of user knowledge as well. Researchers will be presenting an initial demonstration version of the smart video search engine at the CeBIT trade fair in Hannover from March 5-9, 2013 (Hall 9, Booth E08). 

Technology learns from users

“NewsHistory provides users with search algorithms, a data model and a web-based user interface so that they can locate identical sequences within various news videos,” explains Patrick Aichroth from Fraunhofer IDMT. He is responsible for coordinating the institute’s R&D work within the EU’s CUbRIK project. Here, researchers are harnessing user knowledge to optimize and extend the capabilities of automated analysis techniques. “The search engine learns from each individual user, allowing it to keep im-
proving search results. Not only does this improve the quality of results, but the re-
sources needed to undertake the analysis are also cut down,” Aichroth continues.

NewsHistory allows each user to add additional information to the results generated by the search engine, including production and broadcast date, sources and keywords for videos. It is also possible to rate the results. Finally, the user’s search itself is a source of information, providing data that is incorporated into the search engine; the metadata of a newly uploaded video, for instance, passes into the database.

“Comparing digital video data online or within video databases is very complex,” ex-
plains Christian Weigel from the Audio-Visual Systems research group at the IDMT. “Videos that share the same content have for the most part been edited, meaning that they are scaled and encoded in a variety of formats. Also, search engines are often unable to distinguish images cropped from a larger picture, lower thirds or the zoom shots so popular with US news channels.”

The demonstration version being presented at CeBIT will investigate how a selection of TV channels have made use of film footage, changed its form and broadcast it. The user interface displays commonalities and appraises them in graphic form. The search itself is conducted either by inputting text or by directly uploading individual video se-
quences. The researchers’ aim is to make the software sufficiently robust that it could also be used in the future to compare the multimedia content found on big online media portals. The scientists do not imagine archivists or journalists will be the only users. “NewsHistory is of particular interest to media and market researchers, say if they want to assess the televised political duels coming up this year,” concludes Weigel.