12 | Cyclops: Designing an eye-controlled instrument for accessibility and flexible use

I have been reading the paper “Cyclops: Designing an eye-controlled instrument for accessibility and flexible use” which is written by William C Payne, Ann Paradiso, and Shaun Kane. In this blog post I will write about the paper and my reflections on the topic.

The Cyclops is an eye-gaze controlled instrument designed for live performances and improvisation. It is specially motivated by a need for expressive musical instruments that is accessible for people that rely on eye tracking for computer access, such as people diagnosed with amyotrophic lateral sclerosis (ALS). The instrument contains a synthesizer and sequencer, and is displayed as a 2D canvas on a screen and controlled with eye-tracking.

There are multiple challenges of varying size when it comes to eye-tracking being the only controller and interaction method.

Precision: Targets need careful placement, spread out from each other and made bigger. Because of this, the amount of input elements that can be displayed on one screen is reduced.

Timing: Getting to a target on screen means keeping your gaze fixed in one spot, and it’s difficult to difficult to control the timing or rhythm of inputs.

Midas touch: A design challenge when creating responsive gaze-only interfaces is when the user is activating controls accidentally when dwell time is reduced. The dwell time varys widely, depending on what kind of task it is, and it is therefore important that different tasks have different dwell time that is matching the task. The dwell time might also vary from person to person, and that might be more difficult to design for.

Other challenges: Varying accuracy and precision across users, trackers and lightning conditions. Usually more accurate around the center of the screen and less accurate around the edges.

From the users perspective, I think it can be challenging to get used to using such interfaces where the only interaction is eye-tracking. On the other hand, the project aims to help out people who are already used to eye-tracking for computer access, and they are probably already used to this way of interaction. I can imagine that one have to stay extremely focused while interacting, because when not focused and thinking on other things the eyes tend to move around. At the same time, it is like learning any instrument – you’ll need practice to be able to master the instrument. It is still important that the instrument ‘helps’ you along the way, by having different dwelling times depending on the interaction and what is the most natural.

The interface is created with some of these challenges in mind. The musical control is split across three screens which reduces cognitive load and possible unintended interactions. There is a section of the screen that is non-selectable representation of the pitch sequence that can be fixated upon without affecting the output in any way. There is room to grow – there are different screens that the user can select depending on their experience

  • William C Payne, Ann Paradiso, and Shaun Kane. 2020. Cyclops: Designing an eye-controlled instrument for accessibility and flexible use. Proceedings of the International Conference on New Interfaces for Musical Expression.

11 | Comment on: “Reactive Video: Movement Sonification for Learning Physical Activity with Adaptive Video Playback”

“Intergenerational Digital Storytelling” | 07

New Feature: Family Health History

In addition to existing features, a new component will be added to the app – Family Health History. This private section will allow users to document and access comprehensive health information about family members, including allergies, intolerances, and more serious conditions such as strokes, cancers, or autoimmune diseases. Access to this feature will be protected by a password, providing users with the option to view this information if desired.

Potential Challenges for me as an Designer:

  1. Designing a User-Friendly Interface: Ensuring that the Family Health History feature is seamlessly integrated into the app’s existing interface without overwhelming users or compromising usability.
  2. Addressing Sensitivity: Designing the interface and user experience in a way that respects the sensitivity of health-related information, balancing transparency with privacy.
  3. User Education: Developing educational materials within the app to help users understand the importance of documenting and reviewing family health history, as well as guiding them on how to use this feature effectively.

Potential Challenges for Developers and Technical Teams:

  1. Data Security: Implementing robust encryption and access control measures to safeguard sensitive health information, ensuring compliance with privacy regulations such as HIPAA (Health Insurance Portability and Accountability Act).
  2. Database Management: Designing a scalable and efficient database structure to store and retrieve health-related data while maintaining performance and responsiveness.
  3. Integration with Existing Features: Ensuring seamless integration of the Family Health History feature with the app’s existing functionalities, such as user authentication and data synchronisation across devices.

Potential solutions to the mentioned problems:

  1. User-Friendly Interface: Conduct user testing and iterative design cycles to refine the interface, focusing on clarity, simplicity, and intuitive navigation. Provide tooltips, tutorials, or guided tours to help users understand and utilize the Family Health History feature.
  2. Sensitivity Considerations: Offer clear privacy settings and consent prompts, allowing users to control who can access their health information. Provide options for users to add notes or context to health entries, encouraging open communication within families.
  3. User Education: Develop informative guides or pop-up messages within the app to educate users about the importance of family health history and how it can benefit future generations. Incorporate interactive elements or quizzes to engage users and reinforce learning.
  4. Data Security: Implement robust encryption algorithms and secure storage protocols to protect health data both in transit and at rest. Conduct regular security audits and vulnerability assessments to identify and mitigate potential risks.
  5. Database Management: Design a flexible and scalable database schema capable of accommodating various types of health information while maintaining data integrity and relational structure. Implement efficient indexing and query optimization techniques to enhance performance.
  6. Integration with Existing Features: Collaborate closely with the development team to ensure seamless integration of the Family Health History feature with the app’s backend infrastructure and frontend user interface. Conduct thorough testing and debugging to identify and resolve any compatibility issues.

10 | Where do I want to go with my research? 🌱

09 | Digital Sustainability: People, Books and Communities 🌱

08 | Escaping the Carbon Tunnel Vision 🌱

Carbon Tunnel Vision by Dr. Jan Konietzko. This graphic visualizes the so called Carbon Tunnel Vision.
Carbon Tunnel Vision by Dr. Jan Konietzko1
  1. Dr. Konietzko, Jan (2022): Moving beyond carbon tunnel vision with a sustainability data strategy.
    URL: https://www.cognizant.com/us/en/insights/insights-blog/moving-beyond-carbon-tunnel-vision-with-a-sustainability-data-strategy-codex7121 ↩︎

What’s Next? : A Journey Into To The Multi-Sensory Experience

Looking ahead, the future of multisensory design appears promising, with emerging trends and technologies poised to redefine immersive experiences. Advancements in virtual reality (VR), augmented reality (AR), and wearable devices are on the horizon, promising even deeper levels of engagement and interactivity. By keeping abreast of these developments, designers can proactively shape the evolving landscape of multisensory design.

The potential of multisensory design is profound, offering designers the opportunity to craft experiences that deeply resonate with users. By delving into user psychology, considering cross-modal associations, ensuring accessibility, and fostering collaboration across disciplines, designers can transcend surface-level interactions and create experiences that captivate and inspire.

However, it’s essential to strike a balance, avoiding overwhelming users with excessive stimuli. The aim is to enhance the experience, not to inundate users with distractions. Each sensory element should serve a distinct purpose, contributing to the overall narrative and usability of the product or service. Regular user testing and feedback collection are crucial to ensuring that multisensory elements positively impact the target audience.

In essence, multisensory design is poised to redefine how users interact with technology and engage with the world around them. By embracing this paradigm shift, designers can create experiences that transcend the ordinary, leaving a lasting impression on users and reshaping industries in the process.

Looking further into the future, the potential applications of digital sensory experiences are vast. Digital malls may offer smell, texture, and even taste to replicate the sensory aspects of physical shopping experiences, revolutionizing online retail. In the food industry, digital taste manipulation could introduce novel taste experiences without health implications. Media and entertainment could undergo a transformation, with films and advertisements offering fully immersive experiences, blurring the lines between reality and virtuality.

Real Life Applications: A Journey Into To The Multi-Sensory Experience

Digital interfaces, multi-sensory design manifests through tactile feedback on smartphones, such as vibrations confirming user actions. Going beyond, haptic feedback technology simulates real-world sensations like pressing a button or feeling fabric directly on touch screens. Retail spaces leverage multi-sensory design extensively, utilizing music genres, lighting, and textured materials to align with their brand identity. Some even incorporate distinct scents for stronger brand association.

Education is also witnessing strides in multi-sensory design, particularly in interactive learning environments that integrate sight, sound, and touch to convey complex concepts more effectively. Museums, for example, increasingly adopt interactive displays, enriching visitors‘ educational experiences through multi-sensory engagement with exhibits.

Case Studies

Let’s begin with Starbucks, where a strategic redesign of their stores prioritized sensory experiences. They introduced softer lighting, cozy seating, ambient music, and the inviting aroma of coffee. The outcome? Heightened customer satisfaction and increased time spent in-store.

Moving on to Apple, their MacBook Pro showcases innovative sensory design with a touch bar that adapts to the user’s current application. This dynamic tactile interaction enhances user immersion by stimulating multiple senses simultaneously.

Airbnb takes a creative approach by integrating sensory elements into guest experiences. Collaborating with scent specialists, they’ve curated distinct fragrances for various destinations worldwide, each triggering unique emotions and memories associated with those locations.

Design is not only a matter of lines and shapes, but a way to explore new innovative ways to introduce technology into everyday life. As such, Sony’s design team has been exploring new forms of communication that integrate into our lifestyles to develop innovative and enriching experiences transforming functional value and innovation into emotional experiences.

The exhibition is composed of five case study rooms. As you journey through each room, ‘Hidden Senses’ will be gradually revealed; from conceptual interaction into examples of contextual application. Each will showcase different ways in which design is applied to technology, to interact with people, products and spaces and create a perceptual experience; from pure sound experiences to new ways of visualizing information through everyday furniture and lighting. The final area brings all the aspects of the exhibition together, proposing a new and unique life space scenario.

These case studies vividly illustrate how sensory design can profoundly enhance user engagement and satisfaction across diverse industries.

  • https://blog.depositphotos.com/multi-sensory-experiences.html
  • https://medium.com/@faria.faria9/the-impact-of-sensory-design-on-user-engagement-and-satisfaction-fdf203409701
  • https://www.dezeen.com/2018/05/02/sony-hidden-senses-exhibition-sensor-technology-milan-design-week/

7 Key Elements for Multi-Sensory: A Journey Into To The Multi-Sensory Experience

In today’s digital age, crafting multi-sensory experiences is a critical aspect of design across various industries. Integrating sight, sound, touch, smell, and taste can profoundly enhance user engagement and immersion. However, achieving the perfect blend of sensory stimuli requires careful attention to various factors. Let’s delve into seven essential elements that shape the landscape of multi-sensory design.

Here are 7 key elements in crafting for multi-sensory experiences:

1. Balance

Balance is essential for effective multisensory design. All the sensory components should work together harmoniously without overpowering the user. Overstimulation can result in sensory overload, which can be unsettling or confusing. A harmonic experience requires careful balancing of the timing, intensity, and context of sensory inputs by designers.

2. Accessibility

Accessibility concerns are also brought up by designing for all senses. Users may not all receive sensory inputs in the same way; some might be limited in one or more senses, while others may have diseases related to sensory processing. Multisensory design needs to be inclusive, offering many ways for people to interact with the senses to those who would not otherwise be able to.

3. Cultural sensitivity

Different cultures can have quite different sensory impressions. Something that smells good in one culture might be offensive in another. In a similar vein, cultural connections with colors can vary greatly. In order to prevent unfavorable reactions and guarantee a great experience for a wide range of users, multi-sensory design needs to be sensitive to cultural differences and flexible to various groups.

4. Cost and complexity

The cost and complexity of design projects often expand when additional sensory aspects are incorporated. Timeliness and finances may be impacted by the need for more resources, research, and development. The extra cost of developing multi-sensory settings or products needs to be well justified.

5. Technological constraints

Even while technology is developing quickly, there are still limits to the degree to which physical and digital designs can replicate certain sensory experiences. For instance, haptic feedback can somewhat imitate the feel of materials, but it is not able to completely replace the sense of touch. Designers have to work within these limitations or contribute to pushing the limits of what is possible with technology.

6. Sustainability

Multisensory components can affect the surroundings, especially if they incorporate electronics or physical materials. To guarantee that multisensory designs do not unnecessarily harm the environment, sustainability of design materials and product lifecycles must be taken into account.

7. Ethical considerations

Engaging consumers‘ senses has an ethical component. It is the duty of designers to make sure that sensory design is neither misleading nor manipulative. Some unethical techniques include, for example, masking low quality with perfume or masking a lack of functioning with sound.

Crafting multi-sensory experiences is a dynamic endeavor that requires balance, inclusivity, and ethical integrity. By carefully considering factors such as accessibility, cultural sensitivity, and sustainability, designers can create immersive experiences that resonate deeply with diverse audiences. With a commitment to innovation and responsible design, multi-sensory experiences become powerful tools for meaningful engagement and human connection.

  • https://bootcamp.uxdesign.cc/designing-multisensory-experiences-expanding-the-boundaries-of-user-engagement-f689902c5b47
  • https://blog.depositphotos.com/multi-sensory-experiences.html

Artistic Experiences_2: A Journey Into To The Multi-Sensory Experience

In a world where dining experiences often focus solely on taste, A 3D Journey Through Time stands out as a groundbreaking exploration of the senses. Nestled in the heart of Sydney’s Shuffle Studio in Brookvale, this immersive dining project takes guests on a multisensory journey like no other.

At its core, A 3D Journey Through Time is a fusion of gastronomy, technology, and storytelling. Spearheaded by SixthSense Agency, known for their innovative approach to projection mapping, and curated by chef Davide, this experience transcends traditional dining norms. As guests step into the studio, they are immediately enveloped in a world where past, present, and future converge. The space is transformed into a canvas of dynamic visuals, with light projections dancing across tables and walls, creating an ever-shifting backdrop for the evening’s narrative.

But visuals are just one aspect of the experience. A key element is the integration of sound, adding another layer to the immersive journey. Through carefully curated audio, guests are transported through time, further enhancing the storytelling aspect of the evening.

However, it’s not just about what you see and hear; taste plays a crucial role as well. Chef Davide’s culinary creations are meticulously crafted to complement the multisensory experience. From sesame-crusted tuna to roasted cauliflower, each dish is a symphony of flavors that tantalizes the taste buds and enhances the overall immersion.

What truly sets A 3D Journey Through Time apart is its ability to challenge perceptions and ignite curiosity. By blurring the lines between reality and illusion, guests are encouraged to question their senses and explore the boundaries of perception.

But perhaps the most remarkable aspect of this project is its inclusivity. With options for curated wine pairings, bottomless prosecco, or inventive cocktails, there’s something for everyone to enjoy, regardless of their preferences.

As guests leave the studio, they are left with more than just a full stomach; they depart with a newfound appreciation for the power of multisensory experiences. A Journey Through Time is not just a meal; it’s a journey of discovery, a celebration of innovation, and a testament to the limitless possibilities of blending technology with tradition.

  • https://www.sixthsenseexperiences.com/
  • https://secretsydney.com/immersive-dining-experience/
  • https://www.trendhunter.com/trends/a-3d-journey-through-time