Background music is commonly used in film and television to produce a certain mood, but for editors at broadcasting companies like Südwestrundfunk (SWR), a major television and radio company in Germany, finding the perfect selection can be extremely time consuming.
Recently, SWR realized that computational methods utilizing artificial intelligence could help their editors to more quickly identify suitable music for specific situations. As SWR Director Kai Gniffke and Managing Director Jan Büttner; Baden-Württemberg Minister of Science, Research and the Arts Theresia Bauer; and HLRS Director Michael Resch announced at a press conference held at HLRS on October 24, 2019, the company collaborated with HLRS to develop a software tool it is using to automatically identify similarity between pieces of music. (See the SWR press release here.)
The software offers an easy-to-use graphical interface in which an editor can enter a music title that suits a situation. The tool then searches the SWR music archive for other music titles that are similar. This helps the editors to identify music that is not as commonly used as the initial title, but that creates a similar mood.
Whereas commercial music streaming platforms like Amazon or Spotify rely heavily on user activity and "likes" for their recommendation algorithms, the new SWR tool is unusual in that the analysis is based directly on the digital content of the music files themselves. The algorithm extracts and investigates approximately 700 audio characteristics of each file on a mathematical basis. This includes quantifying music characteristics such as tempo, loudness, complexity, silence, and other factors that give each piece of music its unique character. Once these characteristics are defined for each clip, the algorithm performs a nearest neighbor analysis; that is, it ranks other files according to the similarity of their profiles to the profile of the clip being queried.
Representatives of SWR presented the new music recommendation tool at a press conference at HLRS. (Photo: HLRS)
Because the SWR music archive contains approximately 2 million files, running such a complex analysis on its own computing infrastructure would have required its full capacity for months. Through contacts facilitated by the Wirtschaftsförderung Region Stuttgart, however, the broadcaster and HLRS launched a collaboration to develop the tool.
HLRS supported SWR by running the music analysis on its Hazel Hen supercomputer. The analysis was completed in about a week and a half, a major improvement over the six months it would have taken on SWR's own system. Once the analysis data were returned to SWR, the software developers used the results as the basis for its algorithm.
At the press conference, HLRS Director Michael Resch explained that the collaboration was productive for both sides, particularly because it offered an opportunity to explore new approaches that bring together artificial intelligence, data analytics, and high-performance computing. "One of the roles that HLRS sees for itself in Baden-Württemberg is to enable the solution of particular kinds of problems that people encounter in industry, in society, and in research," he explained. "On the other hand, we also saw that there was a lot that we could learn from our colleagues at SWR, particularly as we are increasingly engaged in the field of artificial intelligence and data analytics."
Minister Bauer praised the collaboration as an example of how investments in high-performance computing technologies and expertise not only accelerates scientific research but also promotes innovation in industry and in other corners of society. "We understood from the very beginning that this facility has to be open for industry and for society to enable them to quickly realize their potential," she explained. "These key scientific technologies are of elementary importance in so many disciplines and application areas. Collaborations like this show what this means and makes it accessible for many people for whom the subject of high-performance computing seems abstract."
In a press release, SWR Director Kai Gniffke also celebrated the opportunities that collaboration with HLRS offered the broadcaster: "In order to be at the cutting edge of technical developments, particularly with respect to digitalization, we want to be collaborating with institutes outside of the Funkhaus. The supercomputer of the University of Stuttgart opens an enormous field of possibilities for SWR and catapults us with its enormous computational power into the future. This project is exemplary of how we want to work together with the sciences."