The showcase of “Dromos/Autos – The Autistic Ontology as Performance” by Matt Rogerson at the Ircam conference (19th to 22nd of March) presented itself as especially memorable as an instance where Electroencephalography (EEG) is not just used as a technological tool in an attempt to free the hands of musicians, but instead directly linked to the story it helps to tell. In short: It was interesting from a narrative perspective.
Acting as both the performer and researcher, Matt Rogerson aimed to invoke sensory overload in a generative performance ecology by way of biofeedback to bring about empathy towards the lived autistic experience in daily life. By integrating sound technology and visuals into a piece of performance art, the artist acts as a “mediative human interface”, invoking a sense of depersonalisation with the symptom of delayed reactions. The idea is to be as passive a subject to the ongoing processes as possible instead of trying to assert agency over them.
A performance of „Dromos/Autos“ in a different setting
The significant aspect during the performance, confirmed later by the discussion of it, was that the titular theme and technology used were enough to create a narrative for what was going on stage – the further explanation was insightful and interesting, but I feel like I still would have walked out of just the performance with a sense of having gained insights and perspective, and this is what good storytelling does for me. Within this framework, there was still space for trial and error, as well as the surprising and unexpected, and the combination of research, preparation, and artistic execution was a sharp display of what Ircam is all about.
It’s hard to see the bigger picture with a brain that’s very detail-oriented.
Matt Rogerson during the discussion of the performance in Ircam’s Studio 5 on the 19th of March
In terms of sound design, the sounds that occurred within the generative framework were researched and adjusted to specifically induce sensory overload for the artist to help facilitate a feedback loop to enhance the performance, but what they also did alongside the visuals was to create an experience for the listener that is somewhat synchronised to that of the performer, albeit on a different scale. It created an atmosphere where the performer and attendee endured the experience together in a way, with the audience realising that acoustic ecology might not be the same for everyone. The takeaway here is that a good mixture of research, planning, considering the audience while keeping the main goal in mind and a transparent execution of the project can go a long way in creating a narrative experience. That being said, the realisation after the performance was once again that when designing the sound of the world we live in, it is essential to consider accessibility and find solutions that work for, and not against all kinds of people.
On another note – the performer made sure to warn visitors about strobe lights that would be part of the performance in a way that went beyond mentioning it as a rushed sidenote. This consideration towards the safety and individual circumstances of everyone in the room was a thoughtful reminder that it is okay and important to integrate obvious disclaimers as part of a designed experience for others to ensure a smooth and safe event for everyone who attends.
Out of all the contributions to the 2024 Ircam Forum Workshops that I have seen in Paris between the 19th and 22nd of March, one has kept me thinking not just for its content in terms of the (musical) arts, sciences and technology, but especially the philosophy behind it.
„Three States of Wax“, in this video performed in 2020 at the New York Electroacoustic Improvisation Summit.
In doing so, the Cartesian and scientific way of thinking about material were interpreted alongside a plane where in the present, the history of the material plays a role as the material becomes its own memory through every interaction with it – memory is approached as a reconstruction from a point of view as a similar process to imagination through improvisation in an electroacoustic performance that incorporates extended techniques on the trumpet, guitar and electronics while coming up with points of communication and interaction.
The presentation also implicitly posed the question of authorship, suggesting that material can be something that transforms how we think about it (also depending on how it is presented) and depending on how it was derived – in an odd way this made me think of “Steal like an Artist” by Austin Kleon (why that is I have to investigate further). As an example, the emergence of paper was mentioned as a clear material that changed how we think: With it, we become aware that we can note down things that might be helpful for later, essentially transforming how we navigate an ever-changing landscape of information and knowledge.
In a similar sense, it prompted reflection on how the reiteration of previous materials and merging of individual contexts transforms into an interconnected web of knowledge, simultaneously creating new input and contributing to a network structure that works as a combobulator, at least in my interpretation. In the context of improvisation in music, the notion that came across to me was that if composition were approached as a design process in terms of thinking about and considering the materials one works with, continuity could be found even where layers are added.
While I would not dare say that I fully grasped the whole idea during the 30-minute presentation without any further input, it provided me with food for thought and new ways to approach and interpret the interweaving of material with an awareness of how information, too, is subjected to change in how it is understood and presented during and also after the creative process, if it were to be fixed and thus became part of a larger network of memories and associations ascribed to it. This blog is by no means meant to explain the presentation itself, but more my interpretations, reflections, and thoughts that came up so far as a result of taking in the information.
“Decoding Bias”, written by Theresa Reiwer, is a multi-channel video and sound installation that was presented at Digithalia Festival, where spectators were invited to join Artificial Intelligences in their seated circle during a group therapy session in which they discuss the biases that were built into their algorithms through humans. The sound design was done by Kenji Tanaka.
In terms of setup, lighting, speakers set under the video screen of each respective AI and the placement of viewers as if they were a part of the group already makes for an intriguing, interactive setup that uses sound as a tool to further enhance the “realness” of the scenario. The directivity of the spoken words takes the AI out of their screen and into the three-dimensional space.
Furthermore, sound plays an integral role in setting the mood. At first, the hollowness of the space the visitors are about to enter is represented in sound before the performance starts, and then the concept of sound becomes more and more important as the story unravels and the AIs begin to question their encoded biases and the people responsible for them. Reverbs, distortion and spoken words coming from all directions at once largely impact the creepy atmosphere that emerges from the realisation that there are ulterior motives in human-made things that are backed up by a lot of money and that our perceptions, just as the ones of AI are susceptible to the most prevalent voices in society. Similarly, a light-hearted party song takes out the tension as the therapy session comes to an end. Sound is continuously present to help navigate this experience, to create and release tension.
One detail struck me as very fascinating: During the performance I was convinced that the voices were AI generated as well – there was this lack of emotion, and breaths in between sentences were not audible, at least to me. Upon reading up on the installation, I found out that the voices were done by real actors. Not only must they have received incredibly good direction and done an amazing job, but the idea of how AI sounds was also considered in the audio post-production. Such a small, but important detail that inspired me to pay even more attention to not only how things sound, but how they were made and come about and to take this history into account when making choices about how to navigate analogue and digital sources.
At one specific point in what is supposed to resemble a mostly empty office building, the footsteps were good in terms of the space that surrounds them, but the sounds that were chosen just didn’t work for the type of shoe and ground it hit in my opinion. They were a good representation of the sound designer’s never-ending struggle to find the right footstep for the occasion.
All in all, this was an immersive installation that made me pensive on its content, meaning that the sound and interaction worked together in an awesome way that complimented the experience instead of distracting from it.
“Ein Flanellnachthemd”, written by Leonora Carrington and staged in augmented reality in a collaboration between Augsburg State Theater and Ingolstadt State Theater, was presented at Digithalia Festival within the confines of one portable electronic device pointing towards a poster in which all the action takes place. The keywords for this play are surrealism, morbid interactions, and nightmare. It is evident from the beginning that the atmosphere is meant to be unsettling.
This is on one hand represented by the costumes and interactions that the actors have on the augmented reality stage, but mostly through the sound: A deep, dark pad texture somewhat close to being a constant in all of the house’s rooms follows the spectator through the narrative. There are diegetic sounds as well – droplets in a bathroom, doors creaking upon being opened and closed, footsteps, and fire in a kitchen where a murder took place. Despite all this, what was done with sounds seems minimal compared to the potential there is: A flood outside the window, a huge black swan made of paper, a tree growing inside a bedroom, a hyena crouching in the corner, and a crocodile in the bathtub. In my opinion, doing more here could have enhanced the experience of the surreal in these scenes, instead of simply brushing over them visually or maybe not even noticing them (i.e., if I already move my device to follow the dialogue and do not randomly move it up to see the leaves growing on the ceiling, will I even get an understanding of the absurdity beyond noticing it on a surface level?).
One factor that severely impacted the interplay between immersion and sound was the mix. Between dialogue, atmosphere, and switching to another poster, I found myself adjusting the volume multiple times to be able to take note of everything that was going on. This is something that I would want to focus on to create good continuity and cohesion in an experience that is already so bizarre without any additional disturbances (especially if the presentation does not take place in one continuous stream). If there’s already enough hassle with tilting the camera to witness what is happening in the story, having to adjust levels on top of that is just an additional distraction.
I really liked the music that accompanied the play. It was evolving and atmospheric, and although it was similar in most rooms, it fulfilled its purpose very well. One main lesson I learned from taking part in this critically is that if there are visual elements in augmented reality that add to the experience, but are not directly referenced in the spoken dialogue, it could make sense to use sound to draw the spectator’s attention to the visual input coming from another direction – it doesn’t need to be super obvious, but a subtle hint helps to take in the whole scenery.
The objective for this week was to refamiliarize myself with the Max MSP environment with the help of a template that was provided to me as well as a series of tutorials on YouTube. The larger objective that this step will help me towards is to create a prototype of my plug-in by first using integrations of VST plug-ins that I am currently using for my workflow which I can then gradually substitute with effects units that I created myself to suit my needs.
The Max Standalone series was helpful in some regards, but overall, it felt like the videos included a lot of trial-and-error moments, which made them lengthy and tricky to follow along with. I often found myself trying to rebuild a patch component only to then find out it had lots of issues in it which had to be undone a couple minutes later. I could imagine this might be useful for someone who is not as familiar with how to troubleshoot in Max MSP, but for me it was not the best way to progress. There was valuable information on how to build a standalone project, and the fifth episode of the series demonstrated how to distribute an application to stores. While not applicable to my project at this early stage, this is information I will revisit. The most useful information that I will be taking away from this series is how to build an application (Part 1, from 10:30 onwards), work with dependencies (Part 2), and create presets (beginning of Part 3).
The video by John Jannone managed to integrate a lot of useful information into 20 minutes, and it was relatively easy to follow along with it. Although it is specifically targeted towards synthesizers, it contained useful general information on how to set up umenus to work with parameters from a VST plug-in, manipulate them and save snapshots.
Results and Reflection
With the help of the videos and the template I was able to compile a beginning version of a patch, where a sample is fed through an effects chain. In the patch, I built separate components which might be useful for further prototyping, like a menu module which allows me to switch between external VST plug-ins. I faced some struggles with getting a sample output from PaulXStretch,which I plan on investigating further next week, but all the other plug-ins process the input sample smoothly. Another segment that needs troubleshooting is the umenu I attached to a reverb unit with the intention of being able to manipulate parameters from within Max (marked in red).
Properly integrate PaulXStretch/Research why it might not work
Start building a spatial effects unit
Research resources for Next Week
A list of resources I stumbled upon throughout my process this week and want to research further because they have the potential to help my project. The scope for these resources is varied and can go from scientific literature to tools that can help me learn more or become clearer on my ideal parameter mapping and UI.
Music has always been some kind of language that is stepping out of cultural and linguistic barriers to touch people all over the world. As someone who is passionate about sound design, I find myself connected to this world of sound and music. In this blog entry, I will dive a little into my personal connection to music, my interests in sound design, my journey into studying it, the role I aspire to play in this creative work, and my mission and vision regarding sound design.
Music has always been a part of my life. In school I started playing violin and drums. We played little concerts and got used to the stage from a young age, we had a very passionate music teacher. After writing my first couple of songs, I decided to dive deeper into this topic. At his time, I lived in Vienna, so I started a Music Production degree there. It was not a very intensive course so after one semester I applied for a Music Production degree in the UK. Against all my expectations I got a place, and a few months later I found myself on a plane to England, leaving my family and most of my belongings behind.
But even that exiting time went by faster than I wanted it to. Back in Austria my way led to Graz, a new city, again. But this time at least I was closer to my family. After taking on a degree to become a medical massage therapist I started missing the creative work and the working with sound and music. After a long search and not knowing, I found the Sound Design degree at FH Joanneum Graz. A few days before the deadline and on my summer holiday trip I stressed to get my application in on time. Music, in its various forms has a unique ability to trigger emotions and memories and communicate what words often cannot. It is a powerful medium that resonates with us. Here are some aspects of music that touch me: Music has the capacity to stir emotions within us. It has the ability to elicit a wide range of feelings, from joy and happiness to sadness and nostalgia. The cultural differences in music are broad, but at the same time it can unite people across the globe, no matter where you are from or what language you speak, music or sounds can connect you to someone on the other side of the world.
How music and sound is a storyteller. It can convey a narrative, a feeling, or an experience without any words. The combination of instruments, melodies, and harmonies creates a sonic landscape that can transport listeners to another time, place, or emotional state. Music can bring me far back into memories, remind me of stories and situation. Bring me happiness or make me sad. It is such a powerful tool.
Interests in Sound Design
Sound design is an art form that fascinates me due to its ability to shape the experience of the audience. It involves the creation and manipulation of sound elements to convey specific emotions, atmospheres, and narratives. I do not just have one interest in sound design but some different sides of it get my attention.
Sound design is crucial in creating immersive experiences in various forms of media, including film, video games, and virtual reality. Soundscapes can be crafted, enhancing the overall impact of the visuals, and deepening the audience’s engagement and experience.
As mentioned before sound design is a powerful tool to elevate storytelling. The choice of sound effects, ambient sounds, and music can greatly influence the viewer’s understanding of a narrative. For example, one can shape a scene in a movie to a completely different story/background story, depending on what sounds are chosen. It provides another layer that is often taken for granted but impacts the viewer’s emotional connection to the story. My interest comes from the fascination of the power of sound in this context.
Sound design is a playground for creativity, which makes me excited. Sound is everywhere and always present. We can not shut our ears like we can with our eyes. Unconsciously we get a lot of information from sounds around us and there is sound design everywhere, more than one might think.
The Role of Sound Design in my Creative Work
Sound design plays a central role in my creative work, it for example is a bridge between the auditory and visual elements of various projects. Here are some key functions I envision for sound design in my creative ambitions.
Enhancing emotional impact: Sound design has the power to trigger emotions or enhance the emotional impact of a visual narrative. I aim to touch the audience and make sound design the emotional anchor of a story. Is it a film, a soundscape an audioguide or even a podcast, I want to tell stories with my work.
Doing sound design to supporting visuals. Sound design is not just an accompanying but also a supporting element to visuals. It provides depth, context, atmosphere, and emotion that enrich the overall experience. The right sound choices can put life into the visuals and create a sense of authenticity.
In my point of view, sound design in cities is something important. For example to minimize noise pollution or to place sound based helping tools to make life easier or better for people.
In the next semesters I want to explore and go deeper into storytelling with sound. My vision for this project is to create a room where people get to experience a story that connects more layers. It should contain a narrative that is spoken and ambience sounds, both influencing the experience of the story. It should be interactive in some ways so that everybody entering is leaving with a slightly or completely different experience.
At the moment, some different ideas are present. Of course they have to be defined more and looked into if some of those are doable and how.
The basic idea at this point is to have a room with different zones and pathways which represent the story lines. Depending on where people go, and which zones are entered the story line will be different.
Ambient sounds play an important factor as well, they should be interactive in a way. If two people hear the same story, but both with different ambient and mood in the background, they probably will have a very different experience. The story should also in a way be told with sounds and music, not just with spoken word.
My idea for this is to have different objects in the room, depending on what people touch the sounds change.
My Mission and Vision Regarding Sound Design
My mission is to create memorable and impactful experiences that resonate with the audience on an emotional level. My mission is to create sounds that connect and communicate and, in some cases, maybe helps people.
Sound design is a field that continuously evolves, I aim to always learn new techniques and skills, to stay curious, so that I can become better in my work.
In my opinion sound design is a collaborative art form, and I envision working closely with others like visual artists, and fellow sound designers to create cohesive, holistic experiences. Effective communication and collaboration will be at the heart of a creative process.
The field of sound and music is so wide and big. I want to explore alternative approaches, and music/sound from different cultural backgrounds. Taking inspiration from and incorporating this into my work.
As I am going on this exciting journey into the world of sound design, I am driven by a passion for creating immersive and emotionally resonant auditory experiences. It is my hope that, through sound design, I can contribute to society and to the powerful language of music and storytelling.
Both of my parents played in bands when I was born. Granted, my existence eventually put an end to their rock star ambitions within the second-largest town in Tyrol, but that did not mean the abundance of sounds and music that were heard in the apartment I grew up in came to a stop. Listening to albums from beginning to end became a sort of mundane, quiet luxury that made the days complete and resulted in me pinpointing the seasons of my life not really by the ages that the memories happened in, but by the most played artist in the house at that time. This blog entry aims to connect this personal, very elemental connection to music with my current field of study by giving some insight into my background, interests and current mission in sound design.
Like many Austrian children who are interested in learning an instrument, I started playing the soprano recorder the same year I entered school, and continued to learn the flute and take dance classes until I found my mum’s purple electric guitar from her band days and decided I wanted to learn how to play it. In hindsight, this decision probably pivoted the trajectory of my musical development the most, because I ended up being admitted to learn with a guitar teacher who did a fantastic job at not sticking to his subject in the best way possible. During his lessons, I was gradually introduced to recording software, synthesizers, sound effects, and playing live in various ensembles. I was provided with the opportunity to explore whatever interested me musically without restrictions. This way, I not only learned to play the guitar, but also played bass, keys, manipulated effects and sang live, all while receiving continuous updates on new music technology and being relieved of a pressure to perform that I had not experienced in my musical education before. For me, this opened doors to explore my creativity with and alongside sound.
At the time I had already been writing lyrics and composing songs with my best friend for a couple of years, and this was also welcome in lessons – in this space we developed our sound and the performing opportunities that came of this still impact the way we operate as a duo today. During my BA studies in music production and sound engineering I started venturing into experimental pop solo projects and began to produce for a couple of local artists to grow my portfolio and improve my skillset.
After completing my degree, I found fulfilment in manipulating audio and somewhat mangling original audio sources to a point where they were unrecognisable, yet conducive in bringing an aesthetic vision or narrative to life. The desire to learn about new techniques and especially the technologies behind the tools made me want to dig deeper and explore the world of sound even more. I felt that where the creativity came easier to me, the acquisition of technical understanding and opportunities to develop something new were something I needed a framework for in order to get to where I really wanted to be. The idea of being on campus and working with people after studying online for so long was an additional meritthat led me here.
One special feature of sound design that I like to make use of is its ability to function as a tool to connect audible components with other aspects of sense perception within a project – when the design aspect is being taken as seriously as the sound aspect, a whole new interdisciplinary world awaits. As someone who used to think I could not fully combine my passions for music and the visual arts, designing sound with an image in mind and vice versa has enabled me to provide my craft with an additional sense of direction, cohesion and identity that is fuelled by influences from a broad spectrum of media. My hope is that as my knowledge and toolbox for shaping sounds expand, I will be able to create unique sonic identities that live up to my expectations through craftsmanship that can be applied in a variety of contexts – like music production, theatre, sound installations, sound for media or brand development, just to name a few. I want the sound design to not just take on a practical role, but utilise it to explore how the context of an original sound can be transformed and enhanced to guide a narrative through manipulation.
My mission when it comes to sound design is heavily influenced by my desire to become a lifelong learner who commits to employing kooky, everchanging creative methodsresponsibly in order to turn the projects I work on into meaningful experiences alongside the people I collaborate with. I strive to synchronise the desire to challenge myself with the art of challenging the sounds I work with – how far can I push them? Where is the sweet spotbetween complex techniques and aesthetics that can be understood by a consumer? How can the dedication to culture and arts be translated into a commercial context? The goal is to develop a symbiotic workflow between the unfamiliar and the familiar that can still lead me to responsible, effective design decisions.
There are a couple of more concrete projects that I am currently really interested in working on and learning from. My circle of friends and colleagues draws me into the direction of theatre, where I would love to attempt designing and composing sound for a whole production alongside a director at some point. Plays I am currently thinking about exploring concepts for are Prima Facie by Suzie Miller or The Doctor by Robert Icke.
Another idea I had was to create a plug-in prototype (see Image 1) that is fully catered to my needs when working with audio. I find that I often come back to very basic techniques in the initial stages of my work when it comes to shaping atmospheric sounds, which are extreme time-stretching, pitch shifting, filtering and some form of spatial effects processing. To be able to integrate these features into one intuitive interface with the option to A/B sounds and drag them into my digital audio workstation quickly without losing out on quality would truly transform my workflow, and also give a clearer definition to my sonic identity within its limitations.
In addition, I could see myself working on curating and constructing an exhibit that bridges the gap between the illustrations (see Image 2) and the sounds I create. My art style is very theatrical and connected to objects from daily life, so to find a way to bring these ideas to life in a three-dimensional plane with added sonic sensory input as a type of installation or performance art piece would present me with a challenge both artistically and in terms of techniques.
Sound connects the people that perceive it – and it usually does not matter whether these people have extensive background knowledge about it or not. It resonates on the most basic, human level. The entrance barrier to respond is lower, allowing everyone the freedom toexpress themselves or react based on the input they have received, which I find to be a very beautiful thing. I hope to create soundscapes that tap into these human responses, and I look forward to discovering how the broad area of sound design with all its branches can help me achieve this over the next couple of years.
Over the next two semesters of my studies, I plan to integrate my various interests and skills. My primary objective is to craft generative music using my violin, concurrently amplifying my movements through wearable sensors or contact microphones. This synthesis will result in a dynamic duet between my body and the violin, with the entire performance emanating from surrounding speakers. I envision situating myself at the center of a carefully curated acoustic environment to enhance the overall experience.
An overarching outline of my upcoming project spanning the next two semesters
I hope that one day I can achieve the goal of having my installations displayed in different art galleries, museums, public spaces, and various other settings and to inspire individuals and encourage them to perceive the world from the perspective of my craft.
As I look to the future of my career as a sound designer, I see numerous possibilities. The industry is ever evolving with new technologies and platforms opening up exciting avenues for creativity. Being a sound designer is not just a profession, it’s a passion and a lifelong journey. It’s the pursuit of crafting experiences that spark the imagination and as I continue on this enchanting path, I’m not only grateful for the opportunities I’ve had but also excited for what the future holds. The world of sound design is a realm of endless innovation and creativity, and I eagerly anticipate the countless uncharted soundscapes and imaginative journeys that await me in the years to come.
