by ingrid.hjelm-hansen -

Music jamming as a participatory design method

A case study with disabled musicians. Teodoro Dannemann.

I will use this blog post to write about an article published at the NIME conferences. In 2023, Teodoro Dannemann from Centre for Digital Music on Queen Mary University of London did a case study with disabled musicians. As a part of the course Interaction Design 1, I will discuss my thoughts about his article. 

I chose Dannemanns article because it has a certain relevance to my design and research topic. The aim of the study is to explore the possibility to design new or modified instruments, focusing on a character of disability. The researchers went through three stages to fulfill the needs of musicians with different disabilities: 

  1. A semi-structured music jam session and subsequent analysis

Undergraduate students, researchers and lecturers assisted to arrange jam sessions for four children from the Teletón rehabilitation center (hereafter referred to as the performers). The performers got the opportunity to choose which instrument to play, and worked in teams together with coordinators. Some of the coordinators worked as tempo leaders, and some as normal musicians. The performers actions were observed and recorded. 

In the beginning of the session, each group created a musical score based on a template (see picture below). The performers were welcome to freely create their own unique notation language. The group started the jam based on their home-made score. After a while, the tempo leaders initiated improvisation for ten minutes.

In the end of each round, the facilitator talked to the group to learn what they liked about the jam, what problems they faced, and their overall interaction experience. Then, teams were told to make a slightly more complicated score and start jamming again. This process was repeated three times. 

  1. Obtain individual performers profiles from the data collected

Each group of coordinators discussed the obtained data. Based on the results, they created profiles for each performer comprising the four dimensions: (1) movements and embodiment, (2) musical preferences, (3) difficulties, and (4) capabilities. This helped them organize the collected data and get an overview of the needs of each performer.

  1. Prototype music instruments

A team of designers got the task to elaborate a proposal of one or two instruments for each performer. They based the prototypes on the data gathered in each individual performer profile. The instruments needed to take into account not just the specific needs of each performer and their musical and performance styles, but they also had to be feasible in terms of time (around two months for construction) and budget constraints. Each team got feedback from coordinators and tutors, and iterated the prototypes up to the final delivery event. Each performer was handed their corresponding instrument and got one hour to explore its possibilities. In the end, all participants of the project gathered for a final jam session with additional instruments. 

In the end of the written article, final results are being discussed. They describe two cases of designed instruments, corresponding to two very different performers. The first of the two performers ended up getting a digital cello that could be played with one arm, and a 3D-printed prosthetic forearm. The performer showed a special interest for the prosthesis. However, it was found that the forearm socket needed more fine tuning for them to be able to exert more pressure against the cello. The other performer received a one-handed flute. Even though the performer was really excited about the idea, they were most interested in using the flute to make all sorts of unusual sounds and effects. So, the team ended up making a special „magic flute“. This flute was set up for the performer to easily change how it sounds using a computer, so they could try out different sounds. The design team also put together a simple guide to help the performer get started with fun activities. This would help them focus better on specific tasks and get into a rhythm.

My thoughts

I find the study meaningful and inspiring. It pertains to such an important topic, and the results are effective. I think the article communicated the research and outcomes in an organized manner. Each step in the study is described thoroughly, and the author evaluate how the steps could have been possibly done different. Dannemann is aware of weaknesses in the study, but I also noted down some concerns.  

Firstly, I think the performers should have been involved more in the prototyping phase. This is mentioned by the author in the paragraph: FINAL REMARKS: A CRITICAL REFLECTION. Based on the feedback the performers gave in the delivery event, it becomes clear that there should have been conducted user tests in an earlier stage. Especially concerning the prosthetic forearm that needed more fine tuning. If they tested the prototype iteratively on the real end-user, the prothesis could have worked properly against the cello. Each person with disabilities is different, so it is crucial to involve the designated end-user in an early stage. 

I am also curious about the instrument’s limitations. Does a one arm flute offer the same possibilities as a normal flute? It is designed for such a small user group, and I imagine how difficult it is to create it without any other limitations. It would be interesting to read more about every instrument they made and how they differ from the standards. 

Reference

Dannemann, T. (2023). Music jamming as a participatory design method. A case study with disabled musicians. Centre for Digital Music, Queen Mary University of London. Retrieved from https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://core.ac.uk/download/567597778.pdf&ved=2ahUKEwilpbWjpeKFAxWp4gIHHVSDDEwQFnoECBUQAQ&usg=AOvVaw1JJvVU71_5dazv_tB-4NSt

by david.lasslberger -

#03 Input methods for people with disabilities

In this blog post i want to close off the topics I have had researched a bit before, before starting with completely new research. As the title says, this week’s theme is input methods for people with disabilities., meaning how they are able to use and interact with digital devices such as a PC or smartphone. Since this is a vast and deep topic, I want to focus on an area in which I already have some experience in – gaming. Since I went into the relevance of this particular use case in my first blog post I will not go into it further here. So let’s get right into it!

Depending on the disability, players may not need specialized equipment at all. As is the case with Street Fighter player BrolyLegs, who uses a normal controller to play the fast-paced, fighting game Street Fighter V. Instead of using the controller with his hands, he uses the controller with his mouth. Using this technique, he can go toe to toe with other pro players.

However, sometimes more specialized equipment may be needed. One option to help disabled gamers play their favorite games is to redesign the controller they are using. Both Sony and Microsoft have done just that and developed controllers for their respective consoles which are designed with disabled gamers in mind. In Sony’s case, they have developed the Access-Controller, as they call it. It features big, customizable buttons and a big joystick. Button remapping is supported in the PlayStation software, allowing players to customize as they see fit. It will release this December at a retail price of 89,99€. On the other hand, Microsoft developed the Xbox adaptive controller. It features two large pads which can receive input as well as a d-pad. Its strength lies in its customizability and ability to work with other devices, such as joy sticks or buttons, which is supported in the software. It is also sold for 89,99€.

Xbox Adaptive Controller
Sony Access Controller

One thing I came across in my research which I found interesting was eye tracking. As for a specific use case, I found EyeMine, a piece of software which can be used to control the popular video game Minecraft with your eyes. However, this software only supports specific devices such as the „Tobii Eye Tracker 5“ which, even on sale, is listed at 223€. This investment cost may serve as a potential barrier of entry for low income households. However, basic eye tracking is achievable at very low cost. Any webcam may be turned into an eye tracker using software like the open source software „Opentrack“. The problem then stems from the available software to utilize cameras. However, there already exist many (free) options and as these open source projects grow, it will become more and more easyily for end users to utilize such technology.

While I was doing my research I also stumbled upon this video by Great Big Story who interviewed streamer and psychologist Perrikaryal (her internet handle). Though not disabled herself, she found an input method which may be useful for disabled people. Using an EEG (ElectroEncephaloGram) she was able to pick up brain signals and translate them into usable input. This means that when thinking of an action, e.g. attack or jump, the input can be mapped so that the game character responds accordingly. However, this is not as easy as it may seem. It took her many hours of deliberate imagination practice for the EEG to be able to pick up the signals. Nevertheless, combining this brain tracking technology with eye tracking, she was able to beat Elden Ring, a game commonly known for its high difficulty. This input method may prove a valid option for people with disabilities.

Another form of input are voice commands. Utilizing any microphone and software, which in many cases is open source or even built into the operating system directly (as is the case with Microsoft’s Speech Recognition) users are able to use their voice to input commands or generate text. This may be very useful for slower paced games, such as turn based games, where the speed in which an input must be made is not that decisive. Speech commands may take longer to perform or take a while to be recognized, and may therefore hinder faster paced games. This does not mean that voice commands cannot be used for such games, but it is a problem to keep in mind.

In summary, there exist various input methods which may provide suitable solutions for disabled gamers. These input methods are not only applicable in the field of gaming, however. They may be used as input methods for digital devices in general. But even just in the field of gaming, the possibility for people to play games and connect to each other, regardless of disability, is of incredible value.

by david.lasslberger -

#01 My research journey on the topic of disability

On my research journey thus far, I have had many different ideas on very diverse topics. But the one idea that was most fleshed out and that resonated most with Fabry and me was on the topic of disability. More specifically, I had two ideas of where this topic could lead:

  • Input methods (in the context of gaming) for disabled people
  • Virtual-Reality therapy for disabled people (or people with disorders)

Both of them touch on the same topic, disability, but each one is distinct and offers a different direction for exploration. Currently, I tend to favor the second idea of doing my thesis on the topic of VR treatment and therapy.

Input methods

Individuals with physical disabilities may face challenges when using electronic devices and digital services that rely on standard input methods. One area of focus within this vast domain of interaction is the gaming landscape. In recent years, there have been numerous innovations in the field of accessibility. For instance, Xbox and PlayStation have created their own unique controllers for disabled people. However, designing for people with disabilities presents several difficult problems that need to be addressed. One of the biggest factors is the availability of customization options. The list of requirements is vast and unique for everybody. Nobody should be unable to enjoy their favorite game just because of a physical limitation of their body. For these people, gaming can often be more than just a simple way to pass the time – it can transport them into different worlds and has the opportunity to change their lives.

VR therapy

Virtual reality therapy is a therapeutic approach which utilizes a virtual reality headset to simulate real time environments in an immersive and controlled way. It can be a great training method for physically impaired people. However, VR therapy may not just be an interesting prospect for dealing with physical problems. VR is also being used in the treatment of psychological and mental problems and disorders. Prominently in the treatment of phobias. The fear of flying, spiders or social anxiety can be treated by VR exposure therapy. One significant advantage VR has over more traditional treatment methods is time and cost efficiency. It is inexpensive to set up and can be adapted to many different use cases with relatively low time and financial investments. Furthermore, the US military uses VR to treat war veterans with PTSD. We are just beginning to tap into the value of VR as a training and therapy method.

#Disability #Accessibility #Training #Therapy #VR #InputMethods

How to tackle this subject & what I learned from the second feedback round

Over the course of my second meeting with Fabry, I was able to glean some valuable insights. For one, I learned that I do not need to have the specific topic nailed down to the last detail yet. Rather, it would be more advantageous to have a broad and open perspective on the topic of disability in general and look into many diverse areas instead of limiting myself right now. For example, instead of focusing on just VR or input methods, I could take a look at the topic of immersive media in general. Furthermore, I realized that this exploration of themes and topics is exactly what these blogs are for. In the upcoming blog posts, I might want to tackle a different topic in each one or discuss an intriguing piece of literature I have come across. I look forward to exploring many ideas in the upcoming blog posts.

Potential research questions

At this stage I do not want to narrow my focus too quickly. Instead I want to explore this diverse field of research further before narrowing it down.  With regard to the two topics proposed, some potential research questions could be as follows:

  • How can VR be used in the treatment of patients (with …)?
  • Which input methods (regarding gaming) exist for people with disabilites?

Moreover there are many sub-questions which will arise:

  • How would a VR experience look like?
  • Why do it like this? Who would use it?
  • What are hardware requirements?
  • What are the results? Can it really help?

Relevance for design field & how can design help with this problem?

Accessibility has steadily gained more and more relevance in many different areas and rightfully so. According to the World Health Organization (WHO), approximately 16% of the global population experiences some form of disability. However, progress in this area is slow and disabled people face numerous challenges in their daily lives. VR therapy can be employed to assist people in training their bodies and minds, aiding in their recovery or strengthening their body and mind. When it comes to mental disorders, VR therapy has the possibility of saving lives. As previously mentioned, it is utilized by the US military to treat veterans with PTSD. Despite this, on average, 17 US military veterans commit suicide every day. A grim number, but it is reality. In light of such statistics, the relevance of such treatment methods becomes apparent. VR therapy offers the opportunity to help a variety of patients with different disabilities at a relatively low cost. As for gaming, it has the power to transport people into different worlds, explore wonderful places and alleviate anxiety. Online gaming can help them socialize, and online gaming communities foster a sense of belonging.

As designers, we have the ability to connect many different fields and strive in these multidisciplinary environments. These skills are applicable in numerous domains, such as designing both physical and digital training environments. Designing these therapeutic processes with a holistic approach in conjunction with the medical assistant, the end user and the technology used. Furthermore, medical programs are often only focused on treating the illness but neglect the UX and usability of the people using the product (as I have learned in talking to my brother – more on that further down). These factors ultimately harm the process and the efficiency of the healing method.

Challenges

A research topic like this poses many challenges that need to be overcome. One of which is getting the right contacts and being able to talk and work with them. This necessitates having access to medical professionals and the ability to interact with disabled individuals, possibly conducting tests. 
Furthermore, getting the right medical information to validate or invalidate potential hypotheses will also be challenging.

Personal motivation

This might be a good point to talk about my personal motivation regarding this topic.
On December 4, 2020, the world for my family and me got turned upside down when we got the news that my brother was involved in a big car accident on the highway from Linz to Vienna. We didn’t know whether he would survive, and neither did the doctors. Hearing his voice again after a month in a coma was a huge relief. Still, the struggle was far from over. My brother had to relearn many things. How to properly move his hands, how to walk and much more. However, after every physical rehabilitation period, you could see the progress he made. One of the techniques used in his rehabilitation was virtual reality therapy. I learned about this subject during my bachelor’s as well, but seeing this relatively new form of therapy used in a real setting and the enjoyment my brother had when using it left a lasting impression on me. It’s still sometimes hard and a challenge, but we are glad every day that we still have him.

As for the gaming topic, the story is far less dramatic. I myself enjoy playing games. A lot. Online, I have seen the struggles of disabled gamers and the challenges they face when they try to play games. Accessibility is (fortunately) becoming a bigger topic in the gaming industry and many new games feature extensive accessibility options. However, progress is slow. What interests me especially is the way of controlling the game, how inputs can be made, and how they are handled. I want to help these people gain more self-confidence and freedom. Everybody should be able to enjoy what they love regardless of the physical limitations they migth have.

Possible solutions to the challenges ahead

As mentioned above, my brother has spent a lot of time in hospitals. During that time, he met many doctors and nurses. These contacts may be able to provide support and help with my thesis. Furthermore, my brother can provide many insightful experiences and may be able to assist as a tester for potential applications. Additionally, a relative of mine works as a neurologist. He might also be open to an interview or have useful contacts.
These are difficult challenges to overcome, but there are also opportunities open for me. The best case would be a cooperation with a medical institution. But what will be really possible? Only time will tell.

Intended steps / methodology

Currently, I am in the process of researching this topic to acquire as much information and learn as much as possible. Deciding how to sort and order this information will be vital. Simultaneously, I am also thinking about how to conduct my future research and the possible outcomes of a project on this topic. Possible steps and results may include:

  • Analysis
  • Prototype (physical, VR environment, etc.)
  • User Testing
  • Survey
  • Guide handbook

Further outlook

As stated above I will continue doing research but with a wider view, being open for any intriguing topics and findings. In the upcoming blog posts, I then want to focus more on specific topics.
Also, to note is that I wrote this blog post quite informally and from a very personal point of view. Further blog posts will likely be more „academic“ and „formal“.

References

controller image: Xbox Adaptive Controller | Xbox
VR-therapy image: What is VR Therapy and 3 Pros of Being an Early Adopter (malonesolutions.com)
WHO statistic: Disability (who.int)
Veteran statistic: Virtual Reality Therapy: PTSD Treatment for Veterans (soldierstrong.org)