Compositional methods and Implementation

Compositional Methods

Before addressing the composition for each scene individually, it is essential to ensure that the game’s soundtrack feels cohesive. A unified musical experience maintains player immersion while balancing it with unique characteristics for each scene to keep the experience interesting. To achieve this, I plan to use common instrumentation across various tracks, incorporating unique elements for each scene.

The soundtrack will blend orchestral and electronic elements. The orchestral components, controlled by MIDI instruments, align with the game’s historical theme and match the not-so-high fidelity of the game’s visuals. This mix of virtual acoustic and electronic instruments enhances the fantasy world of Mechanical God Saga. Matching the music’s fidelity with the visuals is deliberate; it creates a coherent experience and can positively impact the game, although sometimes not matching the music to the fidelity of the visuals can also work. This discrepancy can be used to have a positive effect on the game, uplifting it. This is sometimes used in cases where the games are developed for smaller devices without so much computer power, which is not the case for our game.

For composing specific scenarios, it is crucial to convey the appropriate emotions for each scene, such as peacefulness, relief, or stress. Understanding the scene’s context and potential events is key to achieve this. Triggers for musical changes can happen at any time, necessitating quick synchronization with gameplay. It is imperative to identify every possible moment in the music where a change might be needed and adjust accordingly to maintain immersion.

For example, in the forest scene, different musical changes are applied depending on which section of the song is playing when enemies appear. Figure 1 illustrates the initial mapping to achieve a state of tension for this scenario. Other music themes will require multiple states of tension, depending on the evolving situations. However, for the initial forest scene, only a single state of tension is necessary.

By carefully mapping out these musical changes and ensuring quick adaptation to gameplay triggers, the soundtrack will enhance the overall experience of Mechanical God Saga, creating a seamless and immersive journey for players.

The music structure goes as following:

Intro > Segment A > Segment B > Segment C

And for each section there is a need of mapping the changes, as the next table shows, regarding a first state of tension that we want to portray. Synthesized sounds with a metallic aspect are being considered to be associated with the enemy soldiers as for their metallic vests and its connection to the mystery of the nuclear event. As the structure is not linear, the structure mentioned above refers to the way it was composed. Different sections will be looped depending on gameplay, which doesn’t include the intro.

INTROSegMent ASegment bSegment C
Always the same, only plays once.Increased cutoff frequency for the bass.Replace instrumentation to more synthesized/metallic sounds, pumping effect.Increased cutoff frequency for the bass.
Replace flutes with synthesized trumpets.
Change in the melody.
Replace flutes and violins with detuned synthesized sounds.

Fig.1 Adaptive process for a first state of tension.

As the intro starts during an introductory dialogue, it is not possible to get to the enemies while it is playing, therefore there is no need to map changes to this initial part of the music. The other sections have defined changes that are designed to increase a state of alertness and a sense of danger.

Technical Implementation

Creating adaptive music for Mechanical God Saga requires audio middleware to connect the music to the game, reacting to triggers from the game engine. For this project, we will use FMOD, which can be integrated with RPG Maker, the game’s programming software. While there is no official connection between FMOD and RPG Maker, additional programming through JavaScript will be necessary to establish a functional link. Daniel Malhadas, the game’s creator and programmer, is currently researching and developing this connection.

Within the adaptive system, there are branching and layering needs. As a starting point to understand these systems, the next figure shows a simple system designed for the first forest theme, as there is only one trigger happening, related to a state of tension. The red segments represent the altered music for the state of tension.

Fig. 2 Adaptive mapping system for the initial forest theme.

All transitions and overlay triggers need to be quantized to the BPM of the source cue and have meter and quantization grid setup. All starts and stops should have adjustable fade in and fade out lengths. This will ensure a smooth and quick transition between segments.

For the layering approach, instead of exporting individual layers, I exported each version as a full mix and will blend between them when a trigger occurs. This method addresses the need for continuous changes, such as increasing the bass filter’s cutoff frequency to achieve tension. Ideally, these adjustments would be controlled within the audio engine, but due to software limitations with the virtual instruments used, this is not possible. This approach offers a practical solution to those limitations. FMOD’s built-in parameters, such as equalization curves, pitch, reverb, and delay, will be used to create effects like dulling the music when the player is injured, enhancing immersion with equalization curves and possibly stereo field reduction.

As mentioned before, control knobs—either continuous or stepped— are planned to adjust the music based on the scene’s tension, the protagonist’s position, and the health or success rate during battles. If the game’s programmer can access this knob system, applying the necessary changes will be much easier, simplifying communication between the music and game sections.

Scene Analysis and Adaptive Mapping

How to adapt the music to specific situations of the game is a key element of the research, as well as considering all possible scenarios in order to succeed in supporting every possible situation. In this paper there will be an explanation of the current process, from scene and emotional description to the mapping of adaptive needs and the subsequent changes in the music. This methods are tailored for Mechanical God Saga’s game but can also be applied to other games with similar nature.

Firstly we are gonna analyze the scenes of the game’s first episode, which includes Iridir Forest and Iridir Prison. Story telling should be part of the music as well or support the narrative, therefore it is very relevant to understand what is happening in each of the scenes.

Iridir Forest

  • Scene Description 

This is the first scene of the game where the player has to infiltrate the prison to “set god free” or in other words, execute him. In this scene we encounter ourselves in a forest with a lake. The atmosphere is calm and peaceful until the soldiers appear for a first fight.

  • Story and Gameplay

Our character has the possibility to wander freely in the forest until he is approached by three soldiers who claim that he killed various soldiers of their squad. He had swore allegiance to them at the time, they are part of the “empire”. Then there is the first fight against them. After the battle there will be more soldiers walking around, and there is always the possibility to fight them individually (although sometimes then there appear more soldiers in the fight) if the player chooses to, earning points and getting stronger in case of winning. Later there will be another fight with a so called “beast”, which is stronger than the common soldiers.

  • Music Soundtrack

For this scene I thought appropriate to have two musical pieces, excluding the fighting music, so that the player doesn’t stay on each one for too much time as it can take some time to trigger the next piece.

What should the music portray?

First Song (forest wandering): Peacefulness; sense of having to conquer something; the start of a new journey; fantasy.

Second Song (after winning the fight): Relief; sense of victory; positive feeling;

  • Triggers

First Song

  1. When the first three soldiers appear. Add tension.

Second Song

1. Transitioning to the prison side. Subtle hint of progress, add instrumentation.

2. Getting to the “rug” that leads to the prison, when the first beast figure appears. It should get more tense. But be careful as there seems to exist a possibility of leaving.

3. In case of winning the battle, get back to a less tense and more relaxed version. This segment will end quickly as we transition to the next scene inside the prison.

Iridir Prison

  • Scene Description 

The goal of this location is to rescue an old friend who is inprisioned. It is an underground location, illuminated by old fire lamps. There should be an eerie sense of danger and challenge as it is a location controlled by the empire and anything can happen at any moment. Metal is a relevant element that might be used in the composition as it represents the jails.

  • Story and Gameplay

The inprisioned friend was caught robbing the city’s medicine cabinet to save his mother. He was also being watched as it was claimed that he started a union of workers and incited strikes, which is true. 

At first, when entering the prison, there is the possibility to read a book called “Destructionism and the will to live”, which claims that demons are “man-made” and it reflects on their will to live. Then the prisoner can be found in the next room, and to release him the player has to find a switch in the room to unlock the prison, by solving a small “puzzle”. After this, the prison lock opens and two beasts appear to fight shortly after. This fight is more advanced in terms of strategy, there is more complexity and use of nature elements. 

After the fight there is the possibility of carrying on to other zones, with several more enemies to possibly fight and treasure chests with important items to be collected. There are 5 different zones, and from the last one it is possible to get to the Warden Chambers, where the final beast will be. Here there will be the final “boss” fight of this scene and the music should be different, more complex and perhaps using uncommon time measures. The battle can take until approximately 10 minutes. After the fight there will be another music piece and it should become very triumphant when using the travel circle at the end!

  • Music Soundtrack

For this scene there will be two musical pieces, excluding the fighting music. The first will accompany the player throughout the quest and the other one will represent the success of having accomplished the desired goals.

What should the music portray?

First Song: Eeriness, danger, challenge, use metallic elements to represent the jails.

Second Song: Sense of victory; triumphant, conclusion of an important chapter.

  • Triggers

First song:

  1. Entering Iridir Prison F1 room, as a subtle sign of progress.
  2. Switching the switch that opens the prison. Friend is released. Coloration change, refreshment, higher frequencies.
  3. Beasts appearing after friend is released. More tension.
  4. Entering the second F1 zone. Various fights will happen here, transmit more tension.
  5. Entering Iridir F2, subtle sign of progress.
  6. Entering Iridir F3, subtle sign of progress.
  7. Entering Iridir F4, subtle sign of progress.
  8. Opening the golden chest, subtle sign of progress.
  9. Entering F5 zone, subtle sign of progress.
  10. Entering the Warden Chambers where the final beast is located. Peak of tension, epic.

Notes:

Getting back to previous zones is a necessary step in order to progress within this scene, so there shouldn’t be a musical regression when going back to a previous room. Second song is shorter and has no necessary triggers, aside the one to start it. It should be triggered when getting to the travel circle. Getting to the forest again represents the end of this first episode.

Adaptive Elements

The adaptive elements are triggered depending on control signals from the game. Typical controls signals are mentioned below.

  • Position of the protagonist;
  • Position or distance from the antagonist;
  • Environment: weather/time of the day;
  • Game status (tension/relaxation);
  • Emotional state of the player;
  • Health status of the protagonist;
  • Interaction with Non Playable Characters (NPC’s).

For Mechanical God Saga, the most relevant musical adaptations are linked to the game status and player’s position. These aspects are crucial due to the game’s inherent contrast between tense battle scenes and more relaxed exploratory moments. The music will change as the player moves between different rooms, subtly guiding and marking progress. Additionally, the health status and success of the protagonist in battles will influence the musical adaptation during combat.

Musical elements such as structure, tempo, melodic contour, harmony, and loudness can convey a wide range of emotions to support various gameplay scenarios. To evoke tension, common techniques include increasing tempo, changing time signatures, adding low percussion elements, raising the cutoff frequency of filters, incorporating horn instruments, and using tremolo effects. For instance, Age of Empires enhances tension with low percussion and tremolo strings, while Corrupted Dungeons increases percussion and uses horns when enemies appear.

During battles, injury is often portrayed by a decrease in sensors, which can be represented by dulling the sound, typically using a low-pass filter to make audio elements less distinct. Enhanced breathing and heartbeat sounds can also indicate exhaustion, although these techniques are more effective in a 3D first-person environment and may not be suitable for Mechanical God Saga. Another interesting technique is the reduction of the stereo field as used in Nier: Automata, which will be tested in our game.

For technical implementation, control knobs are planned to adjust the music based on the scene’s tension and the protagonist’s position and also health or success rate during battles. The specific implementation details will be explored in the technical chapter.

Regarding battle outcomes, different musical endings will be triggered depending on whether the player wins or loses. Drawing inspiration from Final Fantasy VII, a victory will be marked by an energetic and triumphant arrangement, while a loss will have a dramatic tone that shifts to a hopeful piece, encouraging the player to try again. This principle will be integrated into our game, as players have the opportunity to instantly retry battles.

Game Research

In this documentation, we take a step further into the music creation for the Role-Playing Game (RPG) Mechanical Saga: Life’s Ultimate Gamble and its adaptive elements. After exploring dynamic implementation techniques in the previous semester, we now apply and tailor them to best serve the gameplay experience. Firstly it is crucial to understand the game’s concept and how it is played, which will be analyzed in this paper. After understanding the game, there is a better foundation on how to go about the music, what qualities it should have, also having in mind that it should ideally sound cohesive throughout the game, maintaining a sense of unity between different scenes.

What is the game about?

 “Mechanical God Saga”is a video game that places a strong emphasis on storytelling, exploration and impactful consequences ofplayer choices. It blends key elements of both Japanese and WesternRPG’s, aiming to revolutionize both genres and craft an immersive experience.

The narrative seeks to explore the themes of totalitarianism, genetic mutation, and the clash between humanity and altered beings in a post-apocalyptic society. It delves into the nature of power, the impact of historical cataclysms, and the potential for coexistence amid oppression and prejudice.

Centuries after a devastating nuclear event, the remnants of humanity are trapped in a totalitarian society that has emerged from the ruins of their former civilization. Genetic mutations, stemming from the nuclear disaster and historical experiments, have given some insects and animals extraordinary abilities, enabling them to coexist with humans as equals.

Among humans, a specific gene continues to manifest, resulting in physical deformities such as wings or scarred skin, which society deems as demonic. This has led to persecution by the God Emperor, who seeks to enslave or exterminate these individuals as a religious campaign.

Amid this turmoil, a mysterious entity known as „The Mechanical God“ observes the unfolding events, while its presence and connection to the nuclear catastrophe remain enigmatic.

The narrative raises critical questions:

  • What drives the God Emperor’s genocidal obsession?
  • Can humans and these so-called demons ever achieve peaceful coexistence, or will one group always dominate the other?
  • What dark secrets link the Mechanical God to the origins of the nuclear disaster and the current societal conflicts?

Through these inquiries, the story examines the possibility of overcoming deeply ingrained prejudices and the quest for truth in a world scarred by its past.

What is the goal of the gameplay?

As a Role-Playing Game, there is a significant emphasis on battles and the evolution of playable characters. The player is encouraged to continually enhance the fighting abilities of the characters, whether by exploring various scenarios to find better items and equipment or by manipulating the flow of battle to maximize gained experience.

Gameplay also serves the narrative (and vice versa) in the sense that the player will constantly have to make decisions, sometimes in dialogues but often in actions during battles, affecting how the game world reacts to the characters. Therefore, on the journey to becoming increasingly capable in battles, the player will inevitably have to decide what kind of character they are building and what actions they are willing to take to accumulate more power. These decisions will be, but not limited to, based on moral and interpersonal dilemmas, prompting the player to seek a balance between the moral compass envisioned for the main character and their insatiable quest for power: essential to overcome and surpass the challenges presented by the narrative.

How is it played?

There are different ways of interacting with the game:

  • Exploration

The player will physically navigate the character through a series of interconnected scenarios filled with secrets and hidden optional paths. For players who prefer a more straightforward experience, there will be a linear path available to follow from beginning to end. However, players that take the time to explore and carefully consider clues presented through dialogue and visual context will be rewarded with unique items and abilities that, when used strategically, can significantly alter their gameplay experience. 

  • Dialogue and Decisions

Throughout the game, the player will encounter numerous opportunities to interact with other characters and the world by choosing from multiple options. These choices not only affect the overall narrative but also have an impact on how characters and the world react to the player. For example, consistently exhibiting aggressive behavior toward a faction will lead its members to share that information and respond with displeasure. This principle applies equally to villagers, city residents, and even allies.

  • Battles

The game features a complex and challenging battle system. Battles occur in turns, with each character performing their action in sequence.

Unlike other games with more traditional turn-based systems, where battle interaction is limited to choosing options from a menu, in Mechanical God Saga, the player has the possibility to perform a sequence of inputs to maximize damage to the enemy. Similarly, when under attack, the player, by pressing the defense button at the exact moment the enemy attack hits, can defend themselves, reducing damage by 50%. Thus, actions in battle are interactive and dynamic.

When attacking, the player accumulates a combo counter. By successfully defending enemy attacks in their entirety, they can maintain the combo from turn to turn, increasing it each time. Increasing the combo also proportionally increases the damage the player inflicts on enemies. Consequently, by increasing the damage done with each action, the player also proportionally increases the experience points earned when executing their turn. In this way, the defense mechanism is not only suitable for reducing received damage but also for increasing the maximum potential damage to be dealt and for gaining experience points more quickly, efficiently, and intelligently.

This combo counter is retained from battle to battle. This requires the player to always be attentive and carefully execute their actions, even when facing weaker enemies on the path to a challenging enemy, as maintaining this combo can be the difference between victory and defeat.

Afraid to own a body

This block aims to reflect on the Afraid to Own a Body composition by Lena Meinhardt and Eva DörrIt, which is a fixed media composition based on a poem by Emily Dickinson from around 1866. Eight channels were used for the performance, making the whole experience very immersive, and IRCAM’s Rave software was used in order to generate their sound material. Rave offers models for realtime deep learning based processing, and the model they trained for this piece learned both of their voice timbres, melting them together and generating a new modeled voice. The artists even mentioned they wanted to create something that would develop its own “consciousness”, as a reference to the artificial intelligence capabilities of the mentioned engine, and also a result of various hours feeding voice material to it, including a full spectrum of textures and also intelligible material. Max MSP was used for much of the processing, where the various sound files would be fed into the models. Regarding the composition, it was intended to contrast something that develops and then falls apart again, which matched with their use of intelligible sentences from the poem, such as the often heard “I am afraid to own a body” sentence compared to completely unintelligible voice material that was generated by the artificial intelligence. The whole sonic experience really suited the original poem, and it was a good representation of artificial intelligence capabilities applied to sound compositions.

A Generative Audio Synthesis Workshop

This block aims to reflect on the Generative Audio Synthesis Workshop, where various new plugins from Audiolab were presented by Sina Bokesoy. One that specifically caught my attention was the Oceanic plugin, in which VST parameters can be controlled by 5 different superimposed waves. This waves have their motion determined by Gerstner’s trochoidales wave solutions, that model ocean water on a physical basis. The screenshot below represents the main engine of the plugin, which doesn’t generate sound, but can easily be coupled with other sound generating instruments. The resulting sounds were very enjoyable and interesting textures could be created by manipulating the waves direction, steepness and also its length. The engine concept aims to replicate sea motion, this natural phenomena which in itself never sounds boring, as one can listen to it for hours, to synthesizer motion. 

Other plugins were presented, such as the Thermo 2, where sonic textures could be created. The engine has a controlled entropy state which we can alter and increase instability. Fundamental 3 was another great plugin, inspired by Karlheinz Stockhausen, where test equipment related sounds could be used to generate unique synth textures.

Suit Your Body

During the Digithalia Festival – a festival for virtual theatre forms at Schauspielhaus Graz – we had the chance to watch Suit Your Body, a theatre computer game by Frauen und Fiktion (Berlin).

I’m often in places where people talk about „overweight“, where I ask: what „over“ are we talking about? „Over desirable“, „over morally good“, „over-…er“? Why is this „over“ necessary? – Natalie Rosenke

In Suit Your Body, a space is created where all the bodies are welcome. In an ethereal-looking like sauna, the game allows the player to navigate it, while being accompanied by a sonic narrative based on an interview with Natalie Rosenke, an expert on weight discrimination. The excerpt mentioned above represents the core of this project, where a new perspective on body beauty standards is practiced. Lots of questions are asked during the reflective monologue, reflecting on the roots of body discrimination. Why does the word fat have such a bad connotation? Why do we use the words corpulent or curvy instead of fat? The speaker embraces the term, questioning its connotation. 

Regarding the game production, the environment has the dynamic aspect of being able to move around the space, which changes according to day time – there are changes in light and also reflections – and regarding sound there are footsteps triggered when the character walks, as well as various ambient sounds, including water and sauna elements that would be reproduced at specific times, really matching the scenes. The sonic narrative, which was also produced as a radio play, seems to be linear and apart from the interview, there are also beautiful choirs that support the dialogue.

Decoding Bias

During the Digithalia Festival – a festival for virtual theatre forms at Schauspielhaus Graz – we had the chance to watch Decoding Bias, a multi-channel spatial video and sound installation.

The concept of this installation consists in having eight different types of Artificial Intelligences (AI’s) attending group therapy sessions. As the play starts, they encounter themselves in one of these sessions, and begin to express how their days have been, similarly to human therapy sessions. However, as the AI’s are built for different purposes, they all have different backgrounds and often disagree, creating lots of room for discussion which makes up most of the play. With these discussions, they are able to reflect upon the fact that their “thought patterns” come back to their human-made datasets, which in its core are often inaccurate, incomplete or incompatible, failing to represent the entire population. One of the biases that was discussed was regarding gender and racial stereotypes, where one of the AI’s couldn’t consider a black woman for a high status, leading work position, just because of being a black woman. 

Amidst other discussions, desert yoga and dancing sessions, this piece offers a speculative future scenario that enlightens the viewer and suggests reflection on this very present and sensitive topic, that might, or is already, affecting us humans.

Regarding the sound experience, I consider that the piece really benefited from the spatial aspect, as the viewers sit at the middle or next to the AI characters, which are positioned in the form of a circle and each voice is located at the respective AI position, making the whole experience more immersive. The music and sound compositions were also surrounding the viewer, with effects such as delay going through all the speakers and creating a great experience.

Exploration of interactive storytelling

Anna Semmelrath, November 2023

Music has always been some kind of language that is stepping out of cultural and linguistic barriers to touch people all over the world. As someone who is passionate about sound design, I find myself connected to this world of sound and music. In this blog entry, I will dive a little into my personal connection to music, my interests in sound design, my journey into studying it, the role I aspire to play in this creative work, and my mission and vision regarding sound design. 

Music has always been a part of my life. In school I started playing violin and drums. We played little concerts and got used to the stage from a young age, we had a very passionate music teacher. After writing my first couple of songs, I decided to dive deeper into this topic. At his time, I lived in Vienna, so I started a Music Production degree there. It was not a very intensive course so after one semester I applied for a Music Production degree in the UK. Against all my expectations I got a place, and a few months later I found myself on a plane to England, leaving my family and most of my belongings behind. 

But even that exiting time went by faster than I wanted it to. Back in Austria my way led to Graz, a new city, again. But this time at least I was closer to my family. After taking on a degree to become a medical massage therapist I started missing the creative work and the working with sound and music. After a long search and not knowing, I found the Sound Design degree at FH Joanneum Graz. A few days before the deadline and on my summer holiday trip I stressed to get my application in on time. Music, in its various forms has a unique ability to trigger emotions and memories and communicate what words often cannot. It is a powerful medium that resonates with us. Here are some aspects of music that touch me: Music has the capacity to stir emotions within us. It has the ability to elicit a wide range of feelings, from joy and happiness to sadness and nostalgia. The cultural differences in music are broad, but at the same time it can unite people across the globe, no matter where you are from or what language you speak, music or sounds can connect you to someone on the other side of the world. 

How music and sound is a storyteller. It can convey a narrative, a feeling, or an experience without any words. The combination of instruments, melodies, and harmonies creates a sonic landscape that can transport listeners to another time, place, or emotional state. Music can bring me far back into memories, remind me of stories and situation. Bring me happiness or make me sad. It is such a powerful tool. 

Interests in Sound Design 

Sound design is an art form that fascinates me due to its ability to shape the experience of the audience. It involves the creation and manipulation of sound elements to convey specific emotions, atmospheres, and narratives. I do not just have one interest in sound design but some different sides of it get my attention. 

Sound design is crucial in creating immersive experiences in various forms of media, including film, video games, and virtual reality. Soundscapes can be crafted, enhancing the overall impact of the visuals, and deepening the audience’s engagement and experience. 

As mentioned before sound design is a powerful tool to elevate storytelling. The choice of sound effects, ambient sounds, and music can greatly influence the viewer’s understanding of a narrative. For example, one can shape a scene in a movie to a completely different story/background story, depending on what sounds are chosen. It provides another layer that is often taken for granted but impacts the viewer’s emotional connection to the story. My interest comes from the fascination of the power of sound in this context. 

Sound design is a playground for creativity, which makes me excited. Sound is everywhere and always present. We can not shut our ears like we can with our eyes. Unconsciously we get a lot of information from sounds around us and there is sound design everywhere, more than one might think. 

The Role of Sound Design in my Creative Work 

Sound design plays a central role in my creative work, it for example is a bridge between the auditory and visual elements of various projects. Here are some key functions I envision for sound design in my creative ambitions. 

Enhancing emotional impact: Sound design has the power to trigger emotions or enhance the emotional impact of a visual narrative. I aim to touch the audience and make sound design the emotional anchor of a story. Is it a film, a soundscape an audioguide or even a podcast, I want to tell stories with my work. 

Doing sound design to supporting visuals. Sound design is not just an accompanying but also a supporting element to visuals. It provides depth, context, atmosphere, and emotion that enrich the overall experience. The right sound choices can put life into the visuals and create a sense of authenticity. 

In my point of view, sound design in cities is something important. For example to minimize noise pollution or to place sound based helping tools to make life easier or better for people. 

In the next semesters I want to explore and go deeper into storytelling with sound. My vision for this project is to create a room where people get to experience a story that connects more layers. It should contain a narrative that is spoken and ambience sounds, both influencing the experience of the story. It should be interactive in some ways so that everybody entering is leaving with a slightly or completely different experience. 

At the moment, some different ideas are present. Of course they have to be defined more and looked into if some of those are doable and how. 

The basic idea at this point is to have a room with different zones and pathways which represent the story lines. Depending on where people go, and which zones are entered the story line will be different.

 

Ambient sounds play an important factor as well, they should be interactive in a way. If two people hear the same story, but both with different ambient and mood in the background, they probably will have a very different experience. The story should also in a way be told with sounds and music, not just with spoken word. 

My idea for this is to have different objects in the room, depending on what people touch the sounds change. 

My Mission and Vision Regarding Sound Design 

My mission is to create memorable and impactful experiences that resonate with the audience on an emotional level. My mission is to create sounds that connect and communicate and, in some cases, maybe helps people. 

Sound design is a field that continuously evolves, I aim to always learn new techniques and skills, to stay curious, so that I can become better in my work. 

In my opinion sound design is a collaborative art form, and I envision working closely with others like visual artists, and fellow sound designers to create cohesive, holistic experiences. Effective communication and collaboration will be at the heart of a creative process. 

The field of sound and music is so wide and big. I want to explore alternative approaches, and music/sound from different cultural backgrounds. Taking inspiration from and incorporating this into my work. 

As I am going on this exciting journey into the world of sound design, I am driven by a passion for creating immersive and emotionally resonant auditory experiences. It is my hope that, through sound design, I can contribute to society and to the powerful language of music and storytelling. 

Personal background and Future Goals

Francisco Sylla, November 2023

My journey into the captivating realm of sound and music technology began in my early years when I set out to learn to play the guitar and engage in various musical projects. Music, for me, has always held a profound and unbreakable connection, a universal language that transcends the boundaries of language and culture. The ability of music to convey an extensive array of emotions has been a constant source of amazement. It’s a medium that speaks to the soul, making it resonate in ways that words alone can’t achieve.

These initial experiences unveiled the vast and captivating world of sound technology. I began to take an interest in recording and mixing instruments, which made Garage Band a good friend of mine at the time. This gradual convergence of my interests and experiences eventually fueled my desire to pursue higher education in this field.

In 2018, I successfully earned a bachelor’s degree in Music Technology from Lisbon’s Music School, which equipped me with the skills required for creating immersive sound experiences. This program covered a wide range of topics, including recording techniques, audio mixing, acoustics, electronics, and more. My final projects include the development of a distortion pedal, more particularly, a fuzz one, as well as a sound redesign for Nadja Andrasev’s animated short-film, “The Sound of Licking”. Those were both very fun to do, with the first one delving into the electronic engineering side of audio and the latter one particularly capturing my interest due to its creative nature.

Later on I gained practical experience by working on diverse projects, including mobile games, films, and live sound engineering. One of the projects that stands out as a testament to my sound design work is the Mobeybou project, which involves creating mobile storytelling apps for children. Mobeybou focuses on using technology to nurture multiliteracies, collaboration skills, and intercultural sensitivity in children. As a team member, my role involved infusing these mobile storytelling apps with sounds and music that would transport young learners to various parts of the world.

The challenge of representing different cultures through sound and music was both enriching and demanding. It required me to delve into the musical traditions of various regions, comprehend the cultural nuances, and create soundscapes that were not only

engaging but also educational. For instance, in the „Mobeybou in Cape Verde“ app, I had the opportunity to explore the vibrant and rhythmic sounds of Cape Verdean music, aiming to make the experience as authentic as possible for children. Similarly, in „Mobeybou in Brazil”, I crafted a sonic journey through the rich and diverse musical heritage of Brazil.

Furthermore, I co-founded Producteur Collectif, a film production collective dedicated to bringing creative visions to life. Within this collective, I handle the sound aspects, both on set and in post-production. Our journey began in 2020 when a friend, João Lages, and I decided to pursue our shared interest in filmmaking. Over the years, our team has expanded to support various areas such as writing, production design, and overall assistance. Our latest films include „Sons do Vale“ (Sounds of the Valley), which won at the Segredos de um Vale Festival, „Isto não é um Debate“ (This is not a Debate), presented at Braga’s Festival Política and „Cativeiro“ (Captivity), with support from Zet Gallery.

Through my coursework and hands-on projects, I had the opportunity to explore various techniques and tools used in sound design. I learned to create sound effects, manipulate audio to match the mood of a scene, and experiment with synthesizers to produce unique sounds. It was during this time that I realized the profound impact of sound on the emotional resonance of visual media.

My particular fascination lies in sound design for visual media, and I aspire to further improve my skills in this area, encompassing sound synthesis, advanced post-production, and surround sound. These skills can be applied to various fields, such as film, games, or artistic installations. In the realm of film, understanding surround sound systems has become increasingly relevant as they have become very common formats. When it comes to gaming, I am enthusiastic about exploring dynamic ways of implementing music and sound that can adapt seamlessly to the ever-evolving gameplay, enhancing the overall user experience. I’m also passionate about creating music and hope to apply these skills to music production. In terms of music projects, I have had the opportunity to collaborate with groups like Compasso Quat, which I joined in 2012. This project focused on experimenting with various instruments and reinterpreting early recordings of Giacometti and traditional Portuguese chants. I also had the opportunity to play in different ensembles and more recently, I participated in an electronic orchestra project called ODE, which involved sonic exploration using electronic devices.

As I delve deeper into the world of sound technology, I am continually fascinated by its ability to shape and enhance various aspects of our lives. Sound is an essential part of our human experience, capable of evoking emotions, transporting us to different places, and adding depth to our everyday encounters. The ability to manipulate and harness the power

of sound is not only an art but also a science. This is what initially drew me into the world of sound technology, and it’s what continues to inspire me to explore and push the boundaries of what’s possible.

Currently, my professional journey is evolving as I engage in concrete projects developed by close friends. One particularly exciting project is an unnamed video game in an advanced stage of development. I’m interested in composing and producing the game’s musical soundtrack, having an opportunity to explore dynamic implementation techniques. With these techniques, I hope to transform linear compositions into music that adapts dynamically to gameplay, ensuring that the emotion and intensity of the music adjust to specific actions or situations in the game. Wwise, a widely-used program for implementing sound and music in video games, would be a valuable tool for this type of implementation.

Aside from the implementation aspect, the core of this potential research project lies in carefully crafting the music composition and production. This involves creating music for various game elements such as the menu, fight and exploration scenes, and more. I plan to utilize virtual instruments, manipulate sounds, and potentially incorporate live-recorded instruments if they align with the project’s objectives. Exploring different time measures and tempos could also be a key element of the project, serving as a tool to convey a wide range of feelings within the game’s environment.

The (Not So) Brief Introductory Ramblings of a New Sound Design Student

Hannah Albrecht, 4th of November 2023

Both of my parents played in bands when I was born. Granted, my existence eventually put an end to their rock star ambitions within the second-largest town in Tyrol, but that did not mean the abundance of sounds and music that were heard in the apartment I grew up in came to a stop. Listening to albums from beginning to end became a sort of mundane, quiet luxury that made the days complete and resulted in me pinpointing the seasons of my life not really by the ages that the memories happened in, but by the most played artist in the house at that time. This blog entry aims to connect this personal, very elemental connection to music with my current field of study by giving some insight into my background, interests and current mission in sound design.

Like many Austrian children who are interested in learning an instrument, I started playing the soprano recorder the same year I entered school, and continued to learn the flute and take dance classes until I found my mum’s purple electric guitar from her band days and decided I wanted to learn how to play it. In hindsight, this decision probably pivoted the trajectory of my musical development the most, because I ended up being admitted to learn with a guitar teacher who did a fantastic job at not sticking to his subject in the best way possible. During his lessons, I was gradually introduced to recording software, synthesizers, sound effects, and playing live in various ensembles. I was provided with the opportunity to explore whatever interested me musically without restrictions. This way, I not only learned to play the guitar, but also played bass, keys, manipulated effects and sang live, all while receiving continuous updates on new music technology and being relieved of a pressure to perform that I had not experienced in my musical education before. For me, this opened doors to explore my creativity with and alongside sound.

At the time I had already been writing lyrics and composing songs with my best friend for a couple of years, and this was also welcome in lessons – in this space we developed our sound and the performing opportunities that came of this still impact the way we operate as a duo today. During my BA studies in music production and sound engineering I started venturing into experimental pop solo projects and began to produce for a couple of local artists to grow my portfolio and improve my skillset.

After completing my degree, I found fulfilment in manipulating audio and somewhat mangling original audio sources to a point where they were unrecognisable, yet conducive in bringing an aesthetic vision or narrative to life. The desire to learn about new techniques and especially the technologies behind the tools made me want to dig deeper and explore the world of sound even more. I felt that where the creativity came easier to me, the acquisition of technical understanding and opportunities to develop something new were something I needed a framework for in order to get to where I really wanted to be. The idea of being on campus and working with people after studying online for so long was an additional meritthat led me here. 

One special feature of sound design that I like to make use of is its ability to function as a tool to connect audible components with other aspects of sense perception within a project – when the design aspect is being taken as seriously as the sound aspect, a whole new interdisciplinary world awaits. As someone who used to think I could not fully combine my passions for music and the visual arts, designing sound with an image in mind and vice versa has enabled me to provide my craft with an additional sense of direction, cohesion and identity that is fuelled by influences from a broad spectrum of media. My hope is that as my knowledge and toolbox for shaping sounds expand, I will be able to create unique sonic identities that live up to my expectations through craftsmanship that can be applied in a variety of contexts – like music production, theatre, sound installations, sound for media or brand development, just to name a few. I want the sound design to not just take on a practical role, but utilise it to explore how the context of an original sound can be transformed and enhanced to guide a narrative through manipulation. 

My mission when it comes to sound design is heavily influenced by my desire to become a lifelong learner who commits to employing kooky, everchanging creative methodsresponsibly in order to turn the projects I work on into meaningful experiences alongside the people I collaborate with. I strive to synchronise the desire to challenge myself with the art of challenging the sounds I work with – how far can I push them? Where is the sweet spotbetween complex techniques and aesthetics that can be understood by a consumer? How can the dedication to culture and arts be translated into a commercial context? The goal is to develop a symbiotic workflow between the unfamiliar and the familiar that can still lead me to responsible, effective design decisions.

There are a couple of more concrete projects that I am currently really interested in working on and learning from. My circle of friends and colleagues draws me into the direction of theatre, where I would love to attempt designing and composing sound for a whole production alongside a director at some point. Plays I am currently thinking about exploring concepts for are Prima Facie by Suzie Miller or The Doctor by Robert Icke. 

Another idea I had was to create a plug-in prototype (see Image 1) that is fully catered to my needs when working with audio. I find that I often come back to very basic techniques in the initial stages of my work when it comes to shaping atmospheric sounds, which are extreme time-stretching, pitch shifting, filtering and some form of spatial effects processing. To be able to integrate these features into one intuitive interface with the option to A/B sounds and drag them into my digital audio workstation quickly without losing out on quality would truly transform my workflow, and also give a clearer definition to my sonic identity within its limitations.

In addition, I could see myself working on curating and constructing an exhibit that bridges the gap between the illustrations (see Image 2) and the sounds I create. My art style is very theatrical and connected to objects from daily life, so to find a way to bring these ideas to life in a three-dimensional plane with added sonic sensory input as a type of installation or performance art piece would present me with a challenge both artistically and in terms of techniques.

Sound connects the people that perceive it – and it usually does not matter whether these people have extensive background knowledge about it or not. It resonates on the most basic, human level. The entrance barrier to respond is lower, allowing everyone the freedom toexpress themselves or react based on the input they have received, which I find to be a very beautiful thing. I hope to create soundscapes that tap into these human responses, and I look forward to discovering how the broad area of sound design with all its branches can help me achieve this over the next couple of years.