by dragana.maksimovic -

Όλοι: music making to scaffold social playful activities and self-regulation

Within the field of inclusive design, innovative technologies are beginning to appear that are intended to meet the specific needs of various user groups. Presenting „Olly“ – a melodic textile tangible user interface (TUI) designed with autism spectrum children’s requirements in mind. Through play, Olly aims to promote social relationships and sensory modulation, drawing on her observations from working with a group of five kids who all have a strong love for music.

Olly’s physical form is an embodiment of accessibility and inclusivity. Olly provides a sensory-rich experience since it is made of soft, tactile materials and has elastic ribbons with analog sensors integrated in them. Children are free to create music by themselves or while collaborating with others, as each ribbon stands for a different musical instrument.

Olly’s interactive feature has been carefully designed to take into account the sensory needs and social dynamics that are common in youngsters on the autistic spectrum. When in solitary mode, kids can play with individual ribbons to discover songs they like. But the real magic happens when you use more than one ribbon at once, as this produces even more complex harmonies. This thoughtful design decision encourages cooperative play while simultaneously acting as a sensory-based self-regulation tool.

During a five-week testing period at a Special Educational Needs (SEN) school, Olly’s effectiveness was closely monitored, and the results were incredibly encouraging. The happiness of children, demonstrated by their smiles, singing, and prolonged eye contact, is evidence that Olly is successful in encouraging social interaction and sensory control. Teachers praised the interface for being inclusive and having the ability to create a warm, friendly, and accessible space for every child. Teachers‘ and teaching assistants‘ comments emphasized Olly’s significant influence on participants‘ experiences and its critical function in promoting happy and cooperative music-making sessions.

While Olly is a positive step in the direction of using technology to support autistic children, there are certain issues that need to be closely examined. It’s possible that the findings may be limited by the short duration of the trial and the small size of the group testing the sample. Moreover, while the results are encouraging, more research is needed to determine the long-term effects of Olly on the development of social skills and sensory modulation in a variety of circumstances.

Olly is proof of the incredible impact that inclusive design can have in improving the lives of kids with autism. Through the combination of music, touch interaction, and social play, Olly breaks down traditional barriers to provide its users with a peaceful and uplifting experience. As we keep looking into the relationship between technology and inclusion, projects like Olly are signs of innovation that point the way toward a more compassionate and empathetic future.

by molly.gibson -

Interaction Design – EMG Sonification as a Tool for Functional Rehabilitation of Spinal-Cord Injury

I chose this paper as i find very interesting the more experimental applications of technology like Max, and how the principles we are learning in this class can be applicable in a wide range of fields.

The paper is an exploration in to how EMG sonification can be used for rehabilitation. The main idea is that spinal cord rehabilitation is based on a lot of repetetive and boring movements for the user, and it can be hard to stay motivated to keep executing the same movements. The implementation of complex music generation based on muscle signals has two quite interesting possibilities, one is to give feedback to user for motivation, and the other is to give direct audio feedback if the correct muscles are being activated or not. This is particularly interesting if this is later developed to be something patients can use at home after they leave the hospital. The authors discuss this and the importance of continuing to practice after leaving the hospital. The audio feedback can be particularly helpful for younger children who are harder to motivate to continue with the same repetitive movement over and over. A small disclaimer is that this paper only describes the implementation and possible use of a musical feedback for rehabilitation, they do not include research on how effective the approach is, as it is a project still in development.

Another part of this paper i found very interesting is the accessibility of the technology that the authors implement to conduct the research project. Even with my fairly limited knowledge about Cycling 74 Max and computer signal processing I can understand the basics of the implementation they have done. They base the signal processing on a software tool called Body Brian Digital Musical Instrumental that is specifically developed for researcher that do not have a background from neuroscience or signal analysis.

Through this software and their own objects in Max they have created an interaction model, which i think is very interesting. It is a very cool example of how the principles we have learned so far in this class can be implemented in quite complex ways, and use the music and audio generation for quite complex prototypes. And also applications in fields quite far outside what we normally consider in our work. The development of a complex interaction interface only based on sound and muscle signals feels quite futuristic, but it is also understandable how they implemented. The only thing that I miss in the paper is a more information about how the users interacted with and experienced the technology.

by leonie.dunke -

11 | Review: Feel What You Don’t Hear – A New Framework for Non-aural Music Experiences

Short Description

Aoi Uyama, Danny Hynds, Dingding Zheng, George Chernyshov, Tatsuya Saito, Kai Kunze, and Kouta Minamizawa. 2023. 
Feel What You Don’t Hear: A New Framework for Non-aural Music Experiences.

The research aims to enhance the music listening experience by providing audiences with a physical representation of the performer’s introspection, including elements like rests, groove, and rhythm. This involved analyzing the performer’s physiological responses such as breathing and heartbeat and presenting them to the audience alongside the music performance.

Method:
Preliminary experiments were conducted to extract the performer’s introspection, and a device was designed to present it to the audience. Physiological data, including heartbeat and respiration, were recorded during a piano performance of Debussy’s „Clair de Lune.“ The performer also provided subjective interpretations of the piece, which were translated into vibrations and force feedback.

Results:
User tests with seven participants yielded positive feedback, with participants reporting a more engaging experience when using the device. However, some users found the tactile sensations distracting, indicating room for improvement. The performer noted the importance of physical movements during performances and expressed the need for flexible communication methods to convey introspection sincerely. Their future work aims to refine the system based on user feedback and explore its application with a broader range of performers and audience members.

Review

The research presents a promising approach to enriching the music listening experience by allowing audiences to feel the performer’s introspection physically. The listener becomes more deeply engaged in the musical experience and can establish a new connection with the musician through additional feedback. Particularly for those who have never played an instrument, this offers the opportunity to understand what it feels like to make music. It aims to involve the body not only audibly but also sensorially – to comprehend the sensations experienced during music-making.

This approach is certainly not perfect yet, as indicated by the conclusions. Some test subjects were distracted by the vibrations (which translate the physical sensations of the performers into tangible output), as they did not align with their own emotional experiences while listening. However, further approaches to transferring feedback to other output mediums suggest that there are ways to make the experience less intrusive. In particular, the idea of outsourcing the feedback to a medium not directly connected to the body – like the pillow-like object in the sketch – seems promising. This way, the listener can decide when they want to receive non-auditory feedback and when not.

Furthermore, I see great potential in providing deaf individuals with a new way to perceive music. There are already some devices that provide vibration feedback in response to music, and this experiment confirms that there are further possibilities to enhance immersion. If this idea becomes market-ready in the future, I could imagine it being used at concerts to show live audiences how playing different instruments elicits various physical reactions.