by Hannah Albrecht -

“ontextC” – Technical Diary 6

What happened so far?

One thing that I found has helped me quite a lot when building my setup was to study and learn from other plug-in constructions that worked with effects similar to the one I am trying to achieve (there’s lots of them available to download for free) for practice. Of course, you could always just look at the connections and figure out what is going on, but for me personally copying them into a new patch object by object and really having to think about which connection was made why helped significantly improve my general understanding of the Max environment, and how I could best organise my complex, growing patches for my own understanding.

Insight into one of the patches I re-built – here I learned that colouring patcher cables can really be a huge help, especially as patches grow larger and larger. It’s a simple thing really, but it helps!

To get a cross-platform overview of how the problem can be approached, I also looked at some pure data patches and examined what was done differently there.

Here’s a list of the patchers I learned from:

For Future Reference

I found that the block~ object in puredata seemed like a really useful option for working with FFT sizes and especially FFT sizes that are supposed to be changeable through a parameter, so it might be worth looking into a Max equivalent/alternative for this.

Ongoing

I found that if I am to do a version of the patch for the exhibit, I would like to try it with just one or two parameters in order to prevent information overload for the target audience and make the procedure straightforward and easy to understand. I also used my learning experiences to note down GUI designs that I found easy to navigate, and which constructions worked intuitively for me to inform my own GUI once it is time to create that.

Results and Reflection

While studying these patches dedicated to stretching sound, I found a lot of methods and patching ideas to come closer to an extremely time-stretched result – however, I still found that most of the units did sound close enough to what I wanted to achieve for me to adapt them for my prototype, so this will definitely be a priority for the next stage of the project. Nonetheless, this little excursion helped me get to know my preferred Max workflow a lot, helped me to navigate patches made by others better and gave me new perspectives on problem solving and syntax.

Objectives for Next Time

  • look into jitter objects to determine graphical user interface possibilities
  • integrate stretch units into the prototype with working signals
  • research block~ equivalents and alternatives for Max
by Hannah Albrecht -

“ontextC” – Technical Diary 5

What happened so far?

Over the end of the last semester and the summer, implementation became the main topic for the process. I managed to find decent placeholder models for the EQ, pitch shifting and reverb unit in the Max default resource examples: (in the ‚pitch and time‘/‘effects‘ folder, access by right click > object > open.amxd). With these, I did some testing using exports from the original Paulstretch software to make sure the results could work in the context of what I am trying to create.

Although initially I was headed towards just slightly modifying the phase vocoder that is available for Max, I realised that for my understanding of the algorithm and Max itself it might be better to start and troubleshoot from scratch, to get a result that I could fully explain and modify as needed. To do so, I used my Python analysis and the available Github repository to break down the most important steps of the algorithm (to recap in overview terms: Fourier Transform > Windowing Function > Spectral Processing > Inverse Fourier Transform > Interpolation and Smoothing) in terms of understanding, but also mathematically so I would be able to send the signal through the correct processing chain in Max for the output I am looking for. This also required me to go back into my mathematical education a little bit in order to properly understand what I was working with.

Ultimately I aimed for 4 manually changeable parameters for now: Window size (to control spectral resolution), Overlap (to control overlap between windows), Stretch factor (the most important one) and a Smoothing parameter which is supposed to help create a smoother output with some to few artefacts.

For Future Reference

Another new consideration that came up during this process was that it might be useful to have a tuner of some sort integrated tob e able to tune the edited audio as needed for the current project. However, this is not a priority right now.

Ongoing

I am currently also trying to plan first listening experiences, to be able to test my prototype in the future. My supervisor suggested I look into webmushra to set up listening test scenarios, and another idea was to set up a sonified „Find the Mistake“ station at the exhibition so people could playfully get results for me to evaluate, in a less controlled context of course.   

Results and Reflection

The stage of the project I am in right now is not the most rewarding in that I don’t get any immediate results at the moment, as I am setting up and testing the patch based off my notes and the process I noted down fort he audio signal, but I know it is essential to create a sounding prototype and am hopeful that it will pay off. Either way, I have learned a lot about digital signal processing during my research for this phase of the project, which is always useful.

Objectives for Next Time

  • Get sound through the new signal chain
  • Come up with test scenarios and mockups
  • If I get that far: Try to get reloadable presets set up
by Mahtab Jafarzadehmiandehi -

Evaluation of a Master’s Thesis

Title: Performativity and Liveness: Approaches to Performance and Composition in Electroacoustic Music
Author: Leonie Patrizia Strecker
Institution: University of Music and Performing Arts, Graz
Date: September 2023
Field: Computer Music
Degree Name: Master of Arts (MA)

Artistic Quality:
The thesis tackles challenging concepts like performativity and spatial interaction, which Strecker tries to weave into her artistic work. However, the artistic interpretation feels limited to her personal approach, which could be enhanced by looking at other artists’ work for comparison. Examining how different genres approach these themes could offer valuable context and strengthen her own interpretations.

Degree of Innovation:
Strecker’s work has moments of originality, especially in how she applies theoretical ideas to her performances. Yet, the approach stops short of exploring newer methods or current technologies that could deepen the study of “liveness.” For instance, exploring real-time audience interaction or using AI-driven components might take the concept of liveness in a direction that resonates more with today’s audiences. The potential for further innovation is present but could be expanded.

Independence:
The thesis shows independence, especially in how Strecker applies theory to her compositions. However, a more questioning approach to her chosen ideas could have added depth. For example, considering viewpoints that challenge or add complexity to the idea of performativity in electroacoustic music would show a willingness to explore different angles, which could make her conclusions stronger.

Organization and Structure:
The thesis is clearly organized, but sometimes feels compartmentalized. The theoretical and practical sections could be more smoothly connected, creating a flow that lets theory and application reinforce each other. This reorganization might make the work feel more engaging for the reader.

Clarity and Communication:
Strecker’s writing is mostly clear, though some sections are quite dense with academic language, which could be challenging for readers unfamiliar with performance theory. Simplifying some of the complex ideas, or using more everyday examples, would make the thesis accessible to a wider audience. Terms like “corporeal liveness” and “mediatized presence” could especially benefit from being explained in simpler terms.

Scope of the Work:
The thesis stays focused, allowing for a deep dive into its topics, but this focus might be a bit too narrow. Relying mainly on her own compositions restricts the broader applicability of her findings. Looking at other artists’ work, or contrasting her pieces with notable electroacoustic compositions, might offer a more well-rounded perspective. Including recent advances in music technology, like machine learning in live performances, could also give the thesis a more modern edge.

Orthography, Diligence, and Accuracy:
The thesis is carefully proofed, though there are minor inconsistencies in digital citations. Additionally, bringing in newer scholarship, especially recent studies on live electronic performance, would make the arguments feel timely and better connected to current discussions in the field.

Literature:
Strecker has done well covering essential sources, referencing foundational theorists like Erika Fischer-Lichte and Philip Auslander. However, more secondary sources, particularly from fields like media studies or digital performance, could help add new angles to her analysis. Expanding the literature review in this way could show how her work fits into a broader conversation.

by larissa.holweg -

„Was kaufen wir da eigentlich?“

Bachelorarbeit von Frau Helena Seel

Thema: „Beeinflussung der Kaufentscheidung durch den Einsatz von Bildbearbeitung im Bereich Lebensmittel“ 

  1. Gestaltungshöhe: Die Arbeit zeichnet sich durch eine klare Struktur aus. Die grafischen Elemente und Diagramme unterstützen die Analyse der Konsumentenentscheidungen, könnten jedoch visuell ansprechender gestaltet sein, um die Ästhetik der Forschungsarbeit zu erhöhen.
  2. Innovationsgrad: Die Arbeit bearbeitet ein relevantes Thema im Bereich Konsumentenverhalten und Bildbearbeitung. Auch wenn das Thema praxisrelevant ist, bleibt der Innovationsgrad begrenzt, da vorwiegend bestehende Theorien und bekannte Methoden zur Anwendung kommen.
  3. Selbstständigkeit: Die selbst durchgeführte Online-Befragung sowie die Auswertung mittels EFS Reporting zeugen von einem selbstständigen Ansatz und einem klaren Forschungsziel.
  4. Gliederung und Struktur: Die Arbeit ist logisch gegliedert. Der theoretische Hintergrund und die empirische Untersuchung werden klar voneinander getrennt und übersichtlich dargelegt.
  5. Kommunikationsgrad: Die Arbeit ist sprachlich gut verständlich und vermittelt die Inhalte anschaulich. Gelegentlich könnten Fachbegriffe weiter ausgeführt werden, um auch Leser ohne Vorwissen im Bereich Konsumentenverhalten anzusprechen.
  6. Umfang der Arbeit: Mit etwa 50 Seiten entspricht der Umfang einer typischen Bachelorarbeit. Die Arbeit behandelt alle relevanten Aspekte, könnte jedoch um weitere empirische Details ergänzt werden, um eine tiefere Analyse der Ergebnisse zu ermöglichen.
  7. Orthographie, Sorgfalt und Genauigkeit: Die Arbeit ist überwiegend sorgfältig geschrieben, weist jedoch einige Rechtschreibfehler auf.  Stellenweise könnten die Formulierungen jedoch präziser sein, um Missverständnisse zu vermeiden.
  8. Literatur: Die Literatur ist umfassend und deckt die zentralen Werke zum Thema Konsumentenverhalten ab. Einige aktuelle Studien hätten jedoch ein stärkeres Gewicht auf den neuesten Stand der Forschung gelegt.
by Mahtab Jafarzadehmiandehi -

„Body and Violin Fusion“ – Audio Programming & Mapping Process III

So the audio programming process basically took the entire semester, and it continues to be a focus as I refine it to better align with the compositional idea and ensure the patches are as practical as possible.

I started by mapping the sensors in Max MSP. Each sensor can be mapped to seven parameters using MIDI numbers, the 7th parameter is a measure of total acceleration obtained directly from the SOMI-1 receiver. It’s calculated using a Pythagorean formula based on acceleration data, excluding gravitational force to improve accuracy.

Figure I. Movement parameters of SOMI-1 around the X, Y, Z axis

What I learned from the mapping process is that even though I tried to be specific with the movements, generally you cannot isolate just one of these parameters while moving. This is a key difference I need to consider when mapping movement sensors compared to other stable

MIDI controllers. To effectively map the sensors and keep in mind the application of movements, I divided the motions into the 7 parameters for each sensor:

  • Rotation X
  • Rotation Y
  • Rotation Z
  • Acceleration X
  • Acceleration Y
  • Acceleration Z
  • Total Acceleration

   Figure II. An overview of sensor mapping in Max MSP

After completing the initial movement mapping, I began creating patches for the interaction part using the aforementioned list. This process is ongoing and will likely continue until the project’s completion. Meanwhile, a crucial aspect I am keen to focus on this summer is implementing patches to the sensors and testing their compatibility both independently and with the violin.

Throughout my learning, I am aware that due to the violin, my movement mapping is limited, and I am still trying to figure out how I can wear the sensors either on my hands or elsewhere, but it is also clear that the majority of the mapping is for the right hand or for the bowing part. However, there is also the possibility to map the sensors not only to the usual gestures that occur while playing the violin but also to some unconventional movements to trigger certain parameters, which would require more precise mapping in this case. In general, there are a few possibilities for the mapping process that I need to consider and examine thoroughly.

There are several types of mapping strategies that can be employed, regardless of whether the relationship between the control input and the parameter output is linear or non-linear:

One-to-One Mapping: This is the simplest form of mapping where each control or sensor input directly corresponds to a specific parameter or output

Multi-Parameter Mapping: This approach allows a single sensor input to influence several aspects of the sound to control multiple parameters simultaneously or sequentially. There is also this possibility to change the works of the sensors via pedal to have the combination of different tasks for the sensors

What I also have in mind is to avoid counter-intuitive mapping, which involves controlling parameters in unexpected ways, as it adds an element of unpredictability to the performance. I believe this is unnecessary for the project. Instead, my goal for the mapping part is to ensure that my movements have a clear and meaningful relationship with the parameters they control.

by Mahtab Jafarzadehmiandehi -

„Body and Violin Fusion“ – Types of Interactions II

After finalizing the compositional idea, I attempted to compile a list of all the signal processing types and audio effects that I intend to use in my piece. To practice with fixed elements, the aim was to control the parameters rather than randomize them, so that I can bring the project closer to what I expected and make it executable as well. In other words, I determined which aspects should be under my control and which should not.

Throughout the semester, I experimented with these different types of interactions to explore my sound possibilities and predict how they would affect the sounds of the violin.

  • Multiple Loops (evolving musical patterns) On/Off through monitor?
    On/Off via Sensors? It needs more precise and complicated patches, and the patches should track unnormal gesture and not the way I usually play with violin
  • Audio Signal Processing Pitch shifting
    Time stretching Amplitude/Dynamic Reverb/Delay Timbre
  • Microsoft Word – Exposé II.docx
  • Chorus: to create a thicker sound by duplicating the signal and slightly varying the pitch and timing
  • Sound Synthesis Variations Additive synthesis
    FM (frequency modulation) synthesis Granular synthesis
  • Granular Synthesis
    Grain size:     shorter grains noisier and more textural, longer grains are more faithful to the recorded sounds of the buffer Grain Density: higher density sounds thicker and more continuous, lower density scattered texture and more noticeable individual grains Grain shape (windowing function): the envelope applied to each grain to shape its amplitude over time, currently I am using the Hamming window Grain position (start time): the start pointing of each grain within the original audio sample Grain playback direction: forward or backward!
    Grain spatialization: just have an idea that the grains move around the listeners from everywhere like a rain!
    Grain sequencing: different order of the grains’ playback for more chaotic textures Randomize the parameters: Which is not my goal, but it’s also another possibility!
  • Spatialization The goal is to begin with headphones that track movements and initially use a binaural mix. Additionally, I plan to explore spatialization, utilizing various IEM plugins, including those designed for granular purposes.

by Mahtab Jafarzadehmiandehi -

„Body and Violin Fusion“ – Compositional Aspect I

As I started with audio programming, it felt like I walked into an unknown world with many possibilities, and it wasn’t clear to me which direction I should take. I came across a conference proceeding on New Interfaces for Musical Expression by Perry Cook15, which helped clarify my initial steps. In the conference paper mentioned that, “Make a piece, not an instrument or controller.” So, I decided to come up with the idea of a piece or specific musical composition because I didn’t want to create interesting research questions without a real product or future direction.
During the semester, I came up with two compositional ideas, and I aim to work on both of them since the interaction parts between them are not significantly different.

Piece No. I
The first concept comes from my own experience as a classical violinist who transitioned into electronic music. The overall idea involves contrasting aspects of my experience, demonstrating how every action reflects on other elements, and showcasing the complexity of identity. This piece has three sections:

Section I
1. Introduction
2. Then it goes to looping and layering
3. Atonality
4. Dissonant Intervals
5. Full of aggressive bowing
6. Using violin extended techniques
• For the transition part from section I to section II, I will either put the violin aside or have it in my hands but not playing it, instead I will just control the recorded sound and manipulate it with the sensors.
• Then will control the amplitude and do a smooth decrescendo to be prepared for the second section.

Section II
1. After some choreography, build the contrast on top of distorted and fading out loop
2. Play tonal patterns
3. With clear sentences
4. Subtle movements
5. Everything in this section is about contrast
6. Probably no loops at all!

Section III
1. Back to the chaos but this time overlapping and layering consonant intervals and melodies
2. End the piece immediately!
So the concept for this piece is mostly based on multiple loops, evolving patterns, and layering them to create a some sort of polyphony. The goal is to alter these layers through various types of interaction to showcase the role of the sensors. I will also build a contrast between the manipulated sounds and the raw sound of the violin, primarily in section two, and then take ideas from each of the two sections to build the final section based on that, for the conclusion part.

Piece No. II
The second piece also evolves from the first one, but with a difference: we start with the violin just for the beginning, and then the focus is more on the transition from acoustic to electronic sounds.
1. Playing violin melodically and record it into the buffer and create loops
2. Put the violin aside
3. Wear the sensors
4. Start with different types of interactions step by step
5. The end consists of many synth loops blending into each other

by Anna Semmelrath -

Interactive Audio Drama – Progress so far

The work for my interactive audio drama project is running, but also hitting some problems at the moment.

The process of getting the Bluetooth beacons to work is a slow one. Some problems came up, like that the beacons turn themselves off after a while without using. Also, the open-source App which is for receiving Bluetooth data, which I thought I could use, is now not possible to use, as it is only available for windows devices and not android and iPhone. Luckily I can get some help with the coding, as this is not my area of good knowledge. In the best case we would write our own App which is capable of receiving on processing the input from the beacons. And also has an easy user interface to have small interactions via the App, a little bit of gamification, so that the player is able to now and then choose between answers or but in some information like a code word.

The draft of the story plot is ready. The story combines fiction with a bit of history. In short words, it is based in the Dr. Who universe. The player travels back in time, meets Johannes Kepler (who lived in Graz from 1554 to 1600) and has to fix the timeline, cause something is very wrong.

The intro to the story is written in script form already. I am getting help with the writing of the script, so I can focus more on the recording and the sound design.  

I recorded the first part of the intro, to test if the script works like it is and to come up with some sound design ideas.

The next steps are, to find solutions for the beacon problem, record the next track as a test track, and create the map.  

by Anna Semmelrath -

Interactive Audio Drama

How should it work:

The story is intended to be played/experienced interactively to a certain degree. There should be an app, and it will be played using one’s own phone and headphones. There should be about 6 locations (more or fewer if necessary) that are relevant to the story, such as the well, the marketplace, etc. These locations will be displayed on a map, and one can navigate to them. Bluetooth beacons will be placed at these locations to trigger audio when one is close enough. It should be located in Stadtpark and Schlossberg to avoid streets and traffic.

The app will have its own interface for maps. Using the phone’s GPS, one can navigate on the map. The whole story is intended to take place in Graz.

Based on clues from the story, the player will have options to decide whether to go to location A or location B next to hear the next part of the story or get new clues relevant to solving the task.

It should be possible for the audio to be triggered only when it fits the story’s progression. That is, if someone passes by a location too early, and it wouldn’t make sense to hear something yet, it won’t be triggered.

There could also be a time limit to hear something at a certain location. For example, the player needs information from Person X at the well and Person Y at the marketplace, but one of them is about to leave soon, and the other is badly injured (for example), so there is a time constraint, and the player must decide where to go. Based on the story so far, the player might know which person is more relevant.

At the end, if the player succeeds, the story is resolved, they travel back to their time and they have either fixed their time completely, just a little, or not at all.

by Hannah Albrecht -

“ontextC” – Technical Diary 4

What happened so far?

To know where to start modifying the Max phase vocoder, I drew comparisons between the same stretch factors in PaulXStretch and the phase vocoder. To keep the conditions as similar as possible, I changed the FFT size in PaulXStretch to 1024 and turned off all of the other parameters in the processing chain (harmonics, tonal vs. noise, frequency shift, pitch shift, ratios, spread, filter, free filter and compressor), with the expectation that the resulting sounds would just stretch the source sound (a ten second snippet from an acoustic multitrack recording) using the respective stretching algorithm. This would then allow me to hear differences.

When comparing the results, it quickly became evident that while the phase vocoder provided very transparent sounding stretches at lower stretch factors, the aesthetic quality of the Paulstretch algorithm and the smearing it introduces were a) very different sounding and b) more usable for the intended sound design purposes, where especially stretch factors over 10 become interesting and the original sound source becomes almost unrecognisable.

Note: I have now switched to working with the default phase vocoder that comes with Max as a resource example (Max 8 > Show package contents > Resources > Examples > fft-fun > phase-vocoder-example-folder). It has a lot of similar components.

Ongoing

Currently I am in the process of settling on EQ, reverb and pitch shifting modules to use for the prototype. Another more research-based aspect of the project is to figure out how the provided Python code from the old Paulstretch algorithm works, which will hopefully allow me to modify the phase vocoder towards a direction that suits the imagined aesthetic outcomes of ontextC. My supervisors are kindly is helping me with this, since I am not familiar with Python at all.

Results and Reflection

The results of the comparison are useful, because they define the differences that need to be overcome in order to reach the aesthetic results I am looking for with this plug-in. While some of the inner workings of the Paulstretch algorithm still remain unknown as of now, the Python code will hopefully help to figure out what is missing. Furthermore, being able to set the FFT size over 2048 to a value closer to a value along 4400 would be a next step to imitate the workflow that started this project better – the steps that follow will show whether that is a limitation in Max or not.

As a sidenote: The shortcuts CMD + Option + M to open a locked Max patch and CMD + 8 to remove the frame have been proven very helpful.

Objectives for Next Time

  • Prep draft of sound example through all parts of the signal chain -> How does it start, which sound do we want to get to?
  • Check out Phase vocoder template, start to modify parameters in project draft and experiment
  • Settle on other modules in the processing chain

Keep in Mind: Mapping parameters together will become relevant sooner rather than later – it makes sense to research this as well.