By Kathy Ruoran Li, Junior Project Manager, Art-Science
On October 8, 2021, Swissnex in China invited two professors from École Polytechnique Fédérale de Lausanne (EPFL) in the digital humanities field to share their work in preserving the cultural heritage of the Montreux Jazz Festival and beyond, in celebration of the festival’s first edition in China. Prof. Sarah Kenderdine is the Lead of Laboratory for Experimental Museology (eM+) at EPFL, as well as Director & lead curator of EPFL Pavilions. Dr. Alain Dufaux is the Operations & Development Director of EPFL – Cultural Heritage & Innovation Center (CHC), his work is directly linked with the Montreux Jazz Archives and several associated research and innovation projects.
Computational Museology: Interfaces to Cultural Big Data
Prof. Kenderdine has been designing interactive frameworks for public engagement with cultural heritage for 20 years and is one of the leading figures in the field. She was named a “Digital Shaper” by Bilanz, a project that celebrates the 100 people advancing Switzerland on its journey to become a leading global digital hub of innovations, in 2020 and 2021.
Prof. Kenderdine has long been a pioneer in experimenting new ways for audience to interact in museums
In her talk, Prof. Kenderdine introduced three paradigms addressing different aspects of digital museology and its interfaces to cultural knowledge.
Computational Archives: Shifting from Stewardship to Co-production
According to Prof. Kenderdine, at most museums only a fraction of the collections is on display and the majority of the treasures are hidden from view. At Smithsonian it is 2%; at the British Museum, only 0.4%. At least, such is the case in an “orthodox” model of stewardship in which curators decide what the visitors see. With new archival access modalities, the paradigm can now shifts to co-production through a process of recollection, regeneration and reworking.
Jazz Luminaries is based on the constellations of jazz greats from the Montreux Jazz Archive digitization project. The installation cuts, remixes and replays 5400 artists and 13,000 videos from the total archive of 11,000 hours of video, resulting in a “neural net” based on the social network of the artists. The clustering is based on the numbers of time artists played with other artists.
Similar to tuning a radio, visitors can use a spherical interface to browse through the massive audiovisual archives and listen to snippets of the recording. If they hear something they like, they can then “zoom in” to hear the full song and explore more work by the artist.
In a more traditional museum setting, such interactive installations can also be used for opening up the museum storehouse and enabling visitors to interact with its collection. At Museum Victoria in Melbourne, Australia, Prof. Kenderdine created a massive Data Browser for the museum’s over 100,000 objects to be interactively examined at 1:1 scale on a 360-degree display. The Data Browser pulls data from its Content Management System (CMS) and generates a selection within 15 minutes for visitors to examine. Because it operates as a “real-time curating machine”, each selection is unique, presenting visitors with the unexpected.
Deep Mapping and Deep Fakes
The project “Atlas of Maritime Buddhism” is based on the compelling story of the spread of Buddhism from India through the seaports of Southeast Asia and South China Sea. Research team conducted fieldwork in hundreds of sites spread across 12 countries and recorded thousands of locations in ultra-high resolution 3D panoramic and spherical imaging, including ambisonics and sound field recordings.
The work is now presented to the public with an exhibition that has toured Taiwan and Hong Kong and will travel to Chengdu in November. In the two precious stops, the exhibitions included artifacts collected by local museums to go with the digital presentations. Visitors were able to interact with the digital collection through devices such as the projection dome and the rotating human-scale viewing platform.
“Deep Fakes: Art & Its Double” was another exhibition that recently opened at EPFL Pavilions with 21 installations, re-presenting seminal objects of art, architecture, heritage sites of worldwide cultures. It takes up the challenge to pose crucial questions about the potency of digital replicas and opposes the use of “deepfakes” for manipulation and misinformation, and touches on cross-cutting themes of mirror worlds, digital twins, crypto currency, machine intelligence, and so on.
Dr. Dufaux followed the footsteps of Claude Nobs, who founded the Montreux Jazz Festival in 1967. He began recording the concerts in the highest quality in a time that such technologies were not that common, with a vision to archive and broadcast the festival programs to audience across the world.
The digitalization project was initiated in 2007, now Most of the concert have been digitalized and archived at EPFL. Claude Nobs Foundation was founded to manage the archive and preserve them for the next generation. In 2013, the Montreux Jazz Festival Legacy was inscribed on the international register.
The archive includes 14,000 master tapes in 18 different audio and video tape formats. It covers the festivals from 1967 to 2017, which translates to 11,000 hours video recordings and 6,000 hours of audio recordings. Some concerts were recorded using more than one technology, so that the team can experiment the latest technology risk-free. It is worth noting that the video recording were made in HD since 1991, and it would take another 15 to 20 years before the general public can even view HD content on TV.
Uncompressed archive requires tremendous storage space
The objectives of Montreux Jazz Digital Project were two-fold; it not only digitizes and preserves the recordings for the next generations, but also innovates and researches with the digital archive as a reference database.
The actual digitization work were mainly done by specialists. Dr. Dufaux’s team at EPFL played the management role and is responsible for quality control, documentation and storage, and more importantly, valorization, innovation, and education work that involve researchers in academia.
The team was able to achieve live recording of the concert streams and store them on server. Later that evening, the team transcoded the recording in sub formats that were more suitable for public platform. The morning after, an indexing team would segment all the songs. In the afternoon that followed, author’s rights were checked. So in the next evening after the concert, the videos are accessible for audience on public platforms.
Innovative Projects utilizing the Montreux Jazz Archive
As one can see from the above examples, the Motreux Jazz Archive is far more than simple documentation of the past. Dr. Dufaux’s team continue to work with collaborators to experiment new ways to utilize the documentation and keep it fresh. At the end of the lecture, Dr. Dufaux also touched on some interesting ongoing projects, such as experimenting with long-term storage on synthetic DNA.
Forward Thinking
The ongoing global pandemic has accelerated the depth and reach of utilizing digital tools to create new interfaces to cultural data that are more immersive and interactive. Switzerland is one of the pioneers in digital humanities field and has nurtured numerous projects that have garnered inspiring results. Researchers and practitioners such as Sarah Kenderdine and Alain Dufaux are constantly experiment new ways to create new interfaces to cultural data. Behind all the boundary-pushing and inspiring projects and experiments are countless hours of staff work, years of digital curation and archiving, as well as forward-thinking visions.
Click here to watch the recording of the event.