Brett Volker and Steve Milton, founders of sensory experience company Listen, discuss the importance of sonic branding across touchpoints, how to compose environmental sounds that are purposeful instead of pollutive, and what multimodal experiences mean for accessibility.
How did your professional backgrounds lead you to founding Listen?
BRETT VOLKER: We have a shared background in the music industry. I started my career working for record labels about the same time that I realized the internet was going to change everything about that sector. So, I stayed involved in music but diversified and kept educating myself, going back to university to study marketing and brand communication.
Steve Milton and Brett Volker. Photo: Drew Reynolds
STEVE MILTON: I was playing, touring and composing as a musician before. While studying musicology, I started to look at sound design in a more abstract way, trying to unpack how it is that we experience sound installations and how we can design spaces that allow for both artists to express themselves and for audiences to experience that expression in new ways.
BV: With Listen we connected the dots – between the creation and consumption of music and art, and between technology, culture, sound and branding. We pulled the right folks together – creatives, strategists, musicologists, composers, sound designers and technologists – to produce sensory-driven and human-centric experiences for brands and artists. The digital sphere was still upcoming at the time and companies were looking for new ways to connect with their audiences. There was a lot of fragmentation with respect to the way that sound was used on behalf of brands, so we saw an opportunity to apply the same rigorous strategies and approaches that are inherent to visual branding agencies to sonic design. Today, brands reach consumers through a multitude of channels, so sonic identities should be expressed across a multitude of brand touchpoints, too.
For Unmoored, Listen collaborated with artist Mel Chin to ‘flood’ Times Square with a mixed-reality experience. A nautical traffic jam appeared approximately 8 m above the square’s visitors, who could use their mobile phones or a HoloLens headset to experience the work of art.
Has the pandemic impacted the importance of sound for branding? Especially now that sensory-driven experiences are more limited?
BV: The pandemic has sped up certain behaviours, such as the use of smart speakers and conversational AI. Those technologies were already emerging pre-pandemic, but now people are relying on them even more – and not only in their home space. Sound and voice recognition can play a huge role in reinventing retail, for example by decreasing the need for touch.
To emphasize the importance of diversity and inclusion in science, technology, engineering and mathematics, Listen co-created a mixed-reality experience with Microsoft and Dr Mae Jemison, the first woman of colour to go to outer space. Defying Gravity: Women in Space was on show at New York City’s The Intrepid Sea, Air & Space Museum to celebrate Smithsonian’s Museum Day.
And what about the accessibility of certain experiences, now that we are more stuck in place?
SM: Besides the use of sound, it’s becoming more important to build multiple modalities of experiences, both physical and digital, each unique and fostering connection in their own way. It’s something that already came very naturally to us, but providing remote accessibility is now crucial.
BV: To give an example, we’re working with the Greek government and Microsoft on a project around the ancient site Olympia. We’re building three modalities of an experience. One will be an immersive web experience that anyone around the world with a decent internet connection will be able to access. The second will be an augmented reality app, which was originally intended to be site specific but evolved to work anywhere. And the third will be a HoloLens experience that will launch whenever the world is ready. It shows how we can pivot existing assets to be more accessible, but still provide a rich experience. It’s important to think about how we can broaden and open up the cultural wealth of countries and institutions by leveraging technology.
SM: Accessibility is also about limiting the technological barriers, such as having to download an app on your device or requiring a headset. Currently, the experiences that you can have using your phone versus a HoloLens are still very different, but we are looking forward to a world where you can create more heightened experiences through the most easily accessible devices.
Cover and above: Listen’s life-size Kaleidoscope in Amsterdam, built for the launch of Martin Garrix’s photo book, provided fans of the Dutch DJ with an immersive, personalized experience. Their selfies, taken upon entering, were analysed for unique characteristics by Microsoft AI and compared to images with similar characteristics from the artist’s archive, which then appeared inside the installation. Photos: Yannick van de Wijngaert
Mixed-reality experiences can sometimes still be quite disorientating: a problem you aimed to tackle with a sonic identity for Microsoft’s HoloLens. How can sound help users navigate augmented realities?
BV: When HoloLens, a product that brings high-definition holograms to life in mixed reality, came on the market, it was the first of its kind. When you’re interacting with virtual objects within your physical space, you’re going to need some feedback and Microsoft’s design team soon realized that feedback needed to be sound. So that’s what we created. We went over the user journey looking for those key moments where there’s a need for confirmation and feedback and noticed different types of interactions. In line with that, we delivered a suite of sounds that work together but are also categorized into three distinct moments in the user’s experience. Environmental, welcoming sounds that help with navigation, sounds inspired by real-world interactions that connect to button-like selections as users move their hands through the air, and a series of adaptions of Microsoft Windows tones for the operation.
SM: Even through it’s a ‘futuristic’ experience, it shouldn’t sound like Star Wars. The sonic feedback needs to be warm, relatable and really help users through the AR space.
This is a shortened version of an interview that appears in Frame 140. To read the full conversation, get your copy here.
