Approaching Basic Senses Using Interaction: A Journey Into To The Multi-Sensory Experience

In real life, humans interact with objects using their senses. The five basic senses – sight, hearing, touch, taste, and smell – play an essential role when perceiving the world. Multi-sensory design is becoming increasingly important in this age of digital interactions. However, the most unforgettable experiences are usually multi-sensory based. I will discuss exploring the multi-sensory design case studies and artworks during my research. I will analyse how to interact with multiple senses in designing products or in an artistic way and in which ways it is possible.

An example of fully multi-sensory experience can be food! Food is about more than just taste; it is also about the way it looks, smells, feels, and sounds. Combining these sensory inputs results in a unique and memorable experience that everyone can enjoy. Actually, multi-sensory food experiences are growing in popularity, with many food and beverage companies developing innovative products that appeal to multiple senses.

Food is a complete multi-sensory experience.

Another example of multi-sensory design is from Volkswagen. In 2009, Volkswagen undertook a unique project in Sweden by transforming an ordinary subway staircase into a giant piano. Their idea was to get people to abandon the escalator and head for the stairs. Each step became a musical note, and soon everyone was jamming on the stairs. It wasn’t just about the music; It was about feeling and having fun. The project showed that making everyday things like stairs more enjoyable can actually make people choose healthier options. It’s a great example of how using multi-sensory design can turn ordinary things into exciting experiences.

To continue with my research questions, these are the questions that I will work on during my research;

Central Research Questions:

  • How to approach basic senses with using interaction design?
  • How can we transfer real-life analog multi-sensory interaction to the digital?

Sub-questions:

  • What is sensory design?
  • How do you create a sensory experience?
  • What is multisensory design?
  • What is the role of sensory design in making user experience more effective?

Revelance for Design Fields:

The relevance of multi-sensory design is quite wide in design fields. Most of the time, we can associate any design in which the senses are used with sensory design. First, in user experience design, it is about how users establish connections between senses and products, in what ways they interact and possible scenarios. On the other hand, from an artistic point of view, it is possible to see multi-sensory design in different, even multi-sensory forms in installations and artistic works.

Approaches to Solve Issues/Questions:

To adress the research questions, I will work on various approaches;

  • Analyzing case studies of successful sensory interaction design projects
  • Collaborating/Interview with experts in fields such as psychology or design to better understand sensory perception and the role in design.
  • Collecting data on which different areas and how sensory design is used in UX/Design fields.
  • Exploring the development of new technologies and devices that appeal to sensory experiences.

Challenges in Research:

  • Minimizing the subject that has wide and diverse areas.
  • Finding reliable and up-to-date research in this evolving field.

Personal Motivation:

I’m really interested in learning about multi-sensory design, especially how products can connect with our senses. Also I’m excited to examine case studies to understand how designs can go beyond just being useful and actually make us feel something. Looking at artworks in this area inspires and helps me think outside the box when it comes to design. I also wonder how our emotions and behaviors are affected by what we see, hear, and feel. Mixing things like textures, colors, sounds and smells seems like a fun way to get creative with design; creating products that not only work well, but also make people feel something special.

Relevant Institutions:

  • Interaction Design Foundation
  • MIT Media Lab
  • NN Group
  • Research Gate

Preview of Upcoming Research Phase:

Projected steps in the research process

Projected steps in the research process

In the following blogs I will talk about explaning multi-sensory design and the role, delve deeper into the principles of multi-sensory interaction design, analysing successful case studies, and hopefully collaborating with experts in the field.

01 | A brief introduction to Digital Sustainability 🌱

Did you know that 1 hour of 4k video streaming requires as much energy as a small car for a 4-hour drive?1 Or did you know that one tweet on X (former Twitter) generates as much CO2 as a human fart? Digital technologies don’t produce visible pollution like other sectors, for example the oil or food industries. Therefore, a lot of people wrongly assume that it is “clean”, or environmentally less harmful. But in fact, if the Internet was a country, it would be the 4th largest polluter.2 The energy consumption of the internet and digital devices isn’t the only hurdle for digital sustainability. Sustainability also includes accessibility. Digital services should be available for all people, regardless of their hardware, software, language, location or their hearing, movement, sight, and cognitive abilities.

  • What can we as designers do, to tackle such a huge problem?
  • How can we educate the masses about the environmental impact of digital technologies?
  • How can we minimalize the carbon footprint of our work?
  • How can we ensure accessibility to our work for everyone?

Now you might think that any individual effort to reduce the web’s environmental impact is a drop in the ocean. But as designers, we are in a position of relative power compared to other industries. We build products that might be used by thousands, even millions of users. Any improvements we make have the potential for a vast impact when scaled up to that level. Reducing carbon emissions caused by the products we build is an obvious place to start, while also continuing to think holistically and bearing in mind other aspects of digital sustainability.3 

My vision and mission

My vision is a sustainable use of digital technologies, especially in connection with the Internet. I hope to be able to make a positive contribution to this topic, be it in the form of education or the development of a proposed solution for a very specific area of this major problem. I don’t have a specific goal yet, but I will definitely find one during my research, which I will pursue over the next two years. Until then, I will continue my broad research and share interesting facts and helpful resources with you. 

Ecograder by Mightybytes

The first interesting tool I would like to share with you is Ecograder by Mightybytes. Mightybytes has been a pioneer in sustainable digital development for over two decades. The US-based B Corp-certified digital agency has led the way for its peers by championing sustainable web development. In 2013, Mightybytes released Ecograder, a tool designed assess a web page and present a report highlighting aspects that influence the sustainability of that page. So if you have your own website and want to see how sustainable it is, have it tested by Ecograder and please share your results in the comments section.4

Closing quote

Finally, I would like to quote Gerry McGovern from his book „World Wide Waste“:

“Digital is physical. Every byte is supported by an atom. Every single action in digital costs the Earth energy. […] Digital is demanding an increasing share of the Earth’s energy and resources and is a major contributor to the generation of toxic trash, to a culture of disposability, convenience and the most wasteful behavior ever seen in human history. Used wisely, digital could be saving our planet, making things more productive and efficient, and more environmentally friendly, while improving living standards. Right now, however, digital is killing our planet.” (McGovern 2020, p. 9)5

  1. Videostreaming: Energiebedarf und CO2-Emissionen ↩︎
  2. Frick, Tim (2016): Designing for Sustainability. A Guide to building greener digital products & services. ↩︎
  3. Barker, Michelle (2023): Introduction to web sustainability ↩︎
  4. CO2.JS Case Study – Ecograder by Mightybytes. ↩︎
  5. McGovern, Gerry (2020): World Wide Waste. How digital is killing our planet – and what we can do about it. ↩︎