Report | 26. February 2025

Report of the Cultural Research Data Academy Forum "Music – Media – Cultural Heritage: Intersections between data, research and infrastructure"

By Dr. Vincent Fröhlich , Andrea Polywka, M. A. , Katharina Bergmann, M. A. , Dr. Alexander Stark and Dr. Martin Albrecht-Hohmaier

Our forum of the Cultural Research Data Academy took place 6th of February (1 p.m.) to 7th of February (1 p.m.) at KreativInstitut.OWL in Detmold and online via Zoom.

The secure and sustainable handling of research data is a core competence at the intersection of science and cultural heritage. Since 2021, the Cultural Research Data Academy (CRDA) of the NFDI4Culture consortium has endeavoured to promote the development of culture-related data and code competence (data literacy) across institutions and across disciplines. The focus is on computer-aided thinking and data curation skills in the fields of art history, musicology, film and media studies, performing arts and architecture.

In recent years, we have developed networking formats, offered individual consultations and held training courses for these target groups at universities and cultural heritage institutions.

At this year's forum on 6 and 7 February, the fifth of the Cultural Research Data Academy, we focused on the common challenges faced by our communities (film and media studies and musicology) in dealing with multimodal data in the cultural heritage sector.

We were particularly interested in:

• the handling of research data for media and film studies research projects;
• uniform standards for metadata, as well as an overview of heterogeneous film and film music collections, current information on tools and repositories that are essential for the sustainable handling of research data;
• use cases in the field of AI / generative music and media technologies;
• legal aspects (copyright and related topics in connection with complex multimodal/audiovisual and AI-generated data);
• and the exchange between projects with similar research or metadata.

The event was organised as a hybrid lunch-to-lunch forum. At the KreativInstitut.OWL in Detmold, host Guido Falkemeier welcomed us with an introduction to the institute, which is a unique, interdisciplinary association of the Paderborn University, the Detmold University of Art and Design and the Ostwestfalen-Lippe University of Applied Sciences.

Axel Berndt from KreativInstitut.OWL, who stood in for Tobias Matzner who was ill, kicked off the 11 inputs. In his presentation "Deconstruction – Recombination – Order: A short work report", Berndt explained how he proceeded with an AI-supported composition in which the brief was to create a piece of music in the style of Mozart using Markov models. It became clear how many and increasingly precise specifications were required to create a suitable and performable composition, which also raised questions about the authorship of this work.

In the second contribution, "Living Data Labs: Infrastructure(ing) and community building around research data", Andreas Möllenkamp from the University of Hamburg's Science and Innovation Research Department emphasised that the sustainable implementation of a scientific culture around research data is largely dependent on strengthening a vibrant associated community. The research format of the Living Labs served him as a metaphor for the central importance of integrated and sustainable community building of research data infrastructures and their stakeholders – one of the central activities of NFDI4Culture.

Sarah-Indriyati Hardjowirogo from Leuphana University Lüneburg emphasised in her presentation "Sound/Technologies as research data. Challenges of an open sound culture science", how much data, content and communities differ depending on the research data project. It became clear that easier accessibility of data as well as guidelines for standardisation and internationalisation in the handling of data are absolutely desirable on the path to open science in the field of sound studies.

The second panel began with Lorenz Gilli, who presented ways of analysing individual tracks in DJ sets in his input "Audiovisual and textual research data in music and sound analysis using the example of EDM DJ sets". The focus was on the question of how DJs change their raw material in order to build their sets from it. Gilli showed the various possibilities of the ‘Sonic Visualiser’ to compare audio and video analyses of DJ techniques and draw further conclusions.

Sigrun Lehnert from the Otto-Friedrich-University Bamberg presented "On the trail of the ‘typical’: the Sound of ‘Wochenschau’ as a Digital Humanities Project" with a focus on previous results and challenges in the research of music from newsreel programmes. Due to the large amount of digitised music pieces and excerpts, the necessity of a ‘music information retrieval’ became apparent in order to be able to approach the various questions of attention control and the specific tailoring to topics, countries and events of the newsreel broadcasts.

In her input "Chords and algorithms: A project report on the AI-supported indexing of GDR music programmes", Kim Voss from the German Broadcasting Archive – Deutsches Rundfunkarchiv (DRA) presented the approach to media documentation and data analysis of their extensive source material, which Voss’ team aims to capture using as much automation as possible.

The day was rounded with guided tours of the laboratories in the KreativInstitut.OWL, which provided exciting insights into the creative possibilities of this unique building. Damian Dziwis, visiting professor for composition and sound design for digital media at the KIO, not only presented research approaches, but also performed under the title "Live Coding Concert – generative composition to Algorave".

The second day began with the input from Silke Reich from the Goethe University Frankfurt and Dennis Friedl from Paderborn University "Metadata standards for the description of film (music): Splitting & Integration". They reported on the various experiments with the inclusion of metadata in the ‘Korngold-Werkausgabe’ edition project. The team found that both the FRBR (Functional Requirements for Bibliographic Records) and the EDIROM editing software were not sufficiently accurate for the indexing of the film music material and presented their status in the search for annotation tools that satisfactorily solve the challenges of the AV material for the edition project.

Barbara Alge from Goethe University Frankfurt differentiated and problematised the CARE principles in her contribution "Musical ethnographies in the field of tension between Open Access and CARE principles". Referring to the authoritative article by Sabine Imeri and Michaela Rizzoli "CARE Principles for Indigenous Data Governance" (2022), she pointed out the tension between data sovereignty, data protection and data in the sense of FAIR principles and the sometimes too loose handling of the respective terms. She also reflected on the significance and possibilities of the CARE principles in the context of her own ethnomusicological research.

The second panel of the day began with the input "Composition technology and AI – perspectives for music theory" by Henrik Schuld & Birger Petersen from Johannes Gutenberg University Mainz. They reported on the experiment of using computer-aided software to complete a publication of a chorale collection by Johann Jakob Pagendarm from the early 18th century that has survived in only three of four ‘Stimmbüchern‘. The reconstruction process of the missing bass part was organised in three stages using algorithms, AI and the merging and verification of the results.

Johannes Hentschel from the Johannes Kepler University of Linz / Swiss Federal Institute of Technology Lausanne jumped in with his presentation "How Hard Can It Be? Ground truthing Fumi's Hardgroove set for beat inference" for Lisa Rosendahl (Greening DH), who was unable to attend. Inspired by Lorenz Gilli's lecture, he presented the Timeline Annotator with concrete examples for analysing DJ sets. The aim of his considerations based on audiovisual performance material is, among other things, the production of research data for analysing spontaneous decisions when using parallel tracks in DJ sets.

Oliver Vettermann (FIZ Karlsruhe / NFDI4Culture) had the last input with "Training Season – on the Legal Protection of Art and Artists", which used the public discourse surrounding Katy Perry's AI-generated album, among other things, to pursue the question of who the authors of such a work are from a legal perspective. It became clear that the datafication of music creates new forms of presentation and new possibilities that must be dealt with legally and scientifically, among other things by making a stronger distinction between the concept of work and authorship.

The co-spokespersons Andreas Münzmay from the Paderborn University and Malte Hagener from the University of Marburg formulated a wrap-up of the forum in the form of five takeaways.

1. Every case is different, many are similar: The presentations had shown that the individuality of research projects posed a challenge for the handling of data. This is why it is so important to hold discussions about formats, tools and platforms in forums like this one.

2. This is also why Community is necessary and useful, but also makes work. Standards cannot be meaningfully enforced ‘from above’, but are supported by the community.

3. Corresponding to point 1, mixed methods and scalable readings are often the best procedures, so that both the individuality of the projects and data can be taken into account and established procedures can be used. 

4. Open source and open data may not always be perfect, but there is no alternative: networks must be formed, especially in the further development of software and standards; participation in collaborative projects is key. 

5. The rabbit and the snake: Despite or even in view of the rapid developments in the field of AI, communities should proactively explore the scope, forge alliances and support each other.

As could be seen from the reactions and feedback, the event received very positive feedback overall. With regard to the fundamentally hybrid format organised by the Cultural Research Data Academy with the necessary technical effort, the involvement of online participants was explicitly praised. It was criticised that the proportion of input from the media studies community was somewhat limited, not least due to the three cancellations.
We would like to take this opportunity to thank the speakers who spontaneously stepped in for their commitment and the exciting enrichment of the programme.

The presentation slides will be published soon at ZENODO.