by max.fruehwirt -

07 | Beyond the virtual: Motion sickness and other accessibility problems when navigating the virtual realm

What this blogpost is all about

Continuing the notion of my previous blogpost of understanding VR, AR, MR and XR along with their strengths and weaknesses, this post aims to also provide insight into the different problems one may be confronted with when designing for or engaging with these technologies on the regular and how to best solve them. At this point, it is necessary to mention already, that, while this blogpost will provide a broad overview about the different challenges one may face along with some possible solutions, it is in no way a complete guide yet, as my research into this topic is still ongoing and thus, incomplete.

Discomfort, accessibility and other challenges

To provide a widely accessible, engaging and also comfortable and immersive experience it is important to address a multitude of issues that the user may experience, when engaging with VR, AR or MR. By being aware of said issues as well as their implications to the experience and how to solve them, a much more complex and immersive experience can be created.

Motion sickness and other discomforts

Extended exposure to virtual environments, be it via VR, AR or MR, can lead to a wide variety of discomforts, when not handled and prepared for accordingly.

Simulator sickness, which is a special type of motion sickness, occurs, when our visual and vestibular system receive conflicting information. This can happen, for example, when the visual display or overlayed displayed object suggest rapid motion, but the user’s body senses no corresponding movement. The resulting mismatch in information can lead to disorientation, nausea and discomfort.

Furthermore, extended exposure to close-up virtual objects or displays can leade to both eye strain and general fatigue, as the eye has to constantly strain to focus. This strain may be even increased, if the eye has to constantly focus on rapidly moving objects or re-focus due to overlays, interfaces or similar displayed objects moving back and forth or rapidly in and out of focus.

Last but not least, when switching between AR and VR – which may mainly occur when working with MR solutions – ,but also when switching between AR or VR and the real world, user’s may experience a short bout of spatial disorientation. This may happen, when the transition between the virtual and the real environment is either to abrupt or not enough spatial clues are present in either the virtual or the real environment for the user to orient themselves. This can then cause disorientation, dizziness or short emotional distress.

Accessibility and impairments

Another thing to consider when dealing with artificial environments is accessibility for people with certain sensibilities or impairments. To create an inclusive and immersive experience, a number of challenges have to be considered.

As VR and AR experiences currently heavily rely on visual elements and their space cues, it is very easy to exclude users with visual impairments. Whether it be complete blindness, colour blindness or other similar disabilities, a number of things needs to be considered to make the experience as inclusive as possible.
In a similar vain, people with auditory, cognitive or motor impairments may have trouble interacting with the created environment as well. Be it challenging motion-based inputs, complex interfaces, information overload or lack of adaptability, users may find an interaction to be needlessly challenging for them and thus feel excluded from the experience, unless ample consideration is given.
This, of course, is also true for users with special sensory sensitivities, such as to bright lights, loud sounds or intense vibrations.

As such, accessibility needs to always be considered when designing in and for such environments.

Possible solutions and their integration

These considerations may seem challenging and / or limiting at first, however, they can also be understood as a chance to create a more immersive, comfortable and holistic experience for the user. When designing virtual environments, it is important to constantly question oneself, whether or not ample consideration for the described challenges has been given and remember some of the solutions described below.

Comfort and customisation

To increase the user’s long-time immersion, and comfort while doing so, a number of small changes and customisation choices can already make a huge difference.

One such change is in how the software is structured in the first place. To give an example, by including different options to adjust volume or brightness of the headset, but also providing different levels of haptic feedback, the whole experience can be customised to fit people of different sensibilities. In the same vein, allowing for the adjustment of the displayed colours or sounds or adding audio descriptive elements can also provide more inclusivity for people with auditory or visual impairments. Following this approach, the chance of motion sickness occuring can also be decreased, by allowing the user to choose settings such as movement speed, field of view and add comfort vignettes, that allow the user to find settings, that minimise their overall discomfort when being exposed to VR or MR environments for an extended amount of time.

HaptX haptic feedback gloves

Sony’s accessability controller, compatible with the new PS5 and PS VR

Another way to increase comfort would be by making adjustments to the hardware and how the user may engage with it. Using headsets with reduced latency or the currently trending inside-out tracking, for example, can both reduce motion sickness by boosting the sense of being present within the current environment and also reduce the weight and thus discomfort the user is exposed to long-term while interacting with the digital environment, be it AR, VR or MR. Similarly, combining modern headsets with custom accessibility controllers or even haptic feedback gloves, a variety of different modes of interaction can be provided, that could potentially lead to a more inclusive environment. A few solutions hereby would be tracking gestures of different body parts and allowing for gesture or voice controls as alternative input methods for people unable to use more conventional controllers or adjusting the haptic feedback to provide more or less information depending on the user’s other sensory capabilities.

Control and adaptability

To conclude the described solutions, providing the user with a maximum amount of control, thus allowing them to adapt the experience to fit their specific needs, can greatly boost both inclusivity and immersion. This may seem like an obvious conclusion, at a glance, but the fact that there is still new technologies and ways being developed to create an even better immersive experience for the wide variety of user needs hints at the opposite. As such, it is paramount to always consider your targeted end user while developing and frequently question yourself not only how to best experience a created environment, but also the challenges some users may face when doing so and to react accordingly.

Next steps:

  • Look into immersive environments and how to create then
  • Check methods of engagement and interaction within these digital environments
  • Research into immersion and storytelling
  • Look into interactivity in VR, AR and MR
  • Check interaction solutions for VR, AR and MR

Sources:

1. Creed, C., Al-Kalbani, M., Theil, A. et al. Inclusive AR/VR: accessibility barriers for immersive technologies. Univ Access Inf Soc (2023). https://doi.org/10.1007/s10209-023-00969-0 online in: https://arxiv.org/ftp/arxiv/papers/2304/2304.13465.pdf [08.02.2024].
2. n.A.: Why VR (& VR Headsets) Can Cause Serious Eye Strain & Pain, in: NVision, 2023, https://www.nvisioncenters.com/education/vr-and-eye-strain/, online in: https://www.nvisioncenters.com/ [08.02.2024].
3. n.A.: XR Today’s Guide to Stopping VR Motion Sickness, in: XR Today, 2023, https://www.xrtoday.com/mixed-reality/xr-todays-guide-to-stopping-vr-motion-sickness/, online in: https://www.xrtoday.com/ [08.02.2024].
4. n.a.: VR Design Principles, online in: https://viro-community.readme.io/docs/vr-design-principles [08.02.2024].

by max.fruehwirt -

06 | VR, AR, MR, XR – Exploring the future of extended reality and its technologies

What this blogpost is all about

To explore one of my possible research topics further, this blogpost will look into the strengths and weaknesses of virtual, augmented, mixed and extended reality systems as well as their exact definitions and current technological trends, to gain a better understanding of which system / technology to use in future endeavors when going for different experiences.

VR vs AR vs XR/MR – a comparison

To better understand the difference and similarities between the different technologies, it is first important to understand their definitions as well as their strengths and weaknesses. For this reason, a short overview will be provided.

Virtual Reality (VR)

Virtual reality creates a computer-generated environment that immerses the user in a completely digital, three-dimensional space, often experienced through specialized VR headsets, providing a sense of presence and interaction within the virtual world. The user, thereby, is completely seperated from the real world and any stimuli they experience is completely computer generated. As a result, it usually allows for a much deeper immersion than the other solutions researched in this blogpost, but has its own strengths and weaknesses to be aware of.

Strengths:

  • Offers a completely immersive experience, perfect for training, gaming and simulations
  • Can easily create spaces, that are not accessible normally and / or provide space where normally there would be none
  • Can be used in healthcare, especially in therapeutic applications, to provide immersive therapy, exposure therapy, pain management and rehabilitation

Weaknesses:

  • Isolation from real world may cause emotional distress (solitude)
  • Needs special equipment, that may be costly and / or not readily available

Augmented Reality (AR)

Augmented reality overlays digitally created content onto our real world, enhancing the user’s perception of their surroundings by integrating computer-generated information such as images, text, or 3D models into the real-time environment. These are typically viewed through devices like AR glasses, tablets or smartphone screens, though in recent years, more applications have surfaced. Especially in the automotive industry, heads-up displays also make use of AR to display necessary information directly to the driver while projecting said information directly onto the windshield.

Strengths:

  • Real world information overlay, that relays information in real-time and provides additional input
  • Allows for hands-free interaction, which allows the user to engage the digital content while also staying aware in the real world
  • Useful for product visualisation and trying out products before making a buying decision

Weaknesses:

  • Limited field of view, especially on smartphone screens or tablets
  • Mobile dependency means less computing power, limiting display performance and causes need for optimisation

Mixed Reality (MR)

Mixed reality combines elements of both virtual and augmented reality, allowing digital and physical objects to coexist and interact in real-time. It seamlessly blends the virtual and real worlds, and allows switching between them, enabling users to engage with both types of content simultaneously. While this, of course, can provide difficult to understand with at first, it also allows for a much deeper influence of the user’s perceived reality as a result.

Strengths:

  • A high level of versatility, as it combines both VR and AR, it allows for a broader range of experiences to be created
  • Enables both in-room and virtual connection, communication and collaboration
  • Can, same as VR, be used in a wide variety of industries for training purposes, while also allowing for direct testing in the real world in AR

Weaknesses:

  • Different technologies and their implementation can cause performance and optimisation issues, posing technical difficulties
  • Cost of adaption currently still very high, especially when compared to pure VR or AR solutions

Extended Reality (XR)

Finally, extended reality is an umbrella term, encompassing VR, AR, and MR. It referes to the spectrum of immersive technologies, that extend, enhance, or blend reality. XR is a comprehensive term covering the entire range of experiences, from completely virtual to fully augmented. It aims to offer a holistic approach to immersive technologies. As such, it comes with all of the previously strengths and can, if used correctly, mitigate some of their weaknesses. The opposite, when used incorrectly, is also true, however.

Current trends and technologies

While of course, the market around VR, AR, MR and other technological solutions like that is constantly evolving, it is still important to understand the general direction of these developments in order to better understand them and work with them. As such, some of the most important ones will be listed here.

Stand-alone, wireless VR, AR and MR headsets without the need for external trackers:

HTC recently presented their new inside-out tracker, that would allow for inbuilt-tracking on a multitude of existing 3rd party headsets

While some of the previous generation’s systems still require for external trackers / tracking stations to be set up or for the headset to be connected to a PC via cable in order to ensure a smooth and immersive experience, current trends have since begun to diverge from that. The current trend seems to be to offer stand-alone, wireless VR, AR and MR solutions without the need for a seperate computer or similar unit for calculations, that also offers tracking via inbuilt sensors instead of external stations. The tracker shown in the picture above, for example allows for complete controllerless tracking of a person’s arms, legs, torso and even head, all by attaching it to the limb in question and up to 8 trackers can currently be used together to provide a smooth and easy experience.

Hybrid systems:

Both the Apple Vision Pro and the Meta Quest 3 offer hybrid solutions when it comes to VR and AR.

While there are still certainly solutions, that focus on either VR or AR in particular, MR seems to be the much more common trend, with passthrough headsets becoming more and more widespread. This is especially obvious when looking at both Apple’s recently released Vision Pro, but also Meta’s Quest 3.

Wider application in everyday life:

When looking at the adaption rate of VR, AR and MR solutions, it quickly becomes apparent that the possible applications for it have skyrocketed. While originally more of a niche development, nowadays a wide variety of experiences are offered, ranging from immersive nature documentaries, sport events, games, movies and more. Furthermore, with both the increasing variety of headsets to choose from and their increasing calculation power comes a lower price – except for the apple solution, of course – with an also increased quality in the experience, making the technology much more accessible in everyday life.

Next steps:

  • Look further into different XR solutions and their respective issues
  • Research essential tools for creating immersive virtual environments
  • Check methods of engagement and interaction within these digital environments
  • Look into accessibility and how to ensure it
  • Research into immersion and storytelling

Sources:

1. Springer / Gabler.: Virtuelle Realität, in: Gabler Wirtschaftslexikon, n.y.,
https://wirtschaftslexikon.gabler.de/definition/virtuelle-realitaet-54243/ online in: https://wirtschaftslexikon.gabler.de/ [08.02.2024].
2. n.A.: Was ist Augmented Reality?, in: Omnia360, 2020, https://omnia360.de/blog/was-ist-augmented-reality/, online in: https://omnia360.de/ [08.02.2024].
3. n.A.: Mixed Reality: Wenn sich Reales und Virtuelles vermischt, in: Omnia360, 2023, https://omnia360.de/blog/mixed-reality/, online in: https://omnia360.de/ [08.02.2024].
4. n.a.: Extended Reality, in: RyteWiki, n.y., https://de.ryte.com/wiki/Extended_Reality, online in: https://de.ryte.com/wiki/Hauptseite [08.02.2024].
5. Hayden, S.: Vision Pro Teardown Shows Balancing Act Between Cutting Edge Tech & Weighty Design, in: ROADTOVR, 2024, https://www.roadtovr.com/apple-vision-pro-teardown-ifixit/, online in: https://www.roadtovr.com/ [08.02.2024].
6. Hayden, S.: Quest 3 Teardown Shows Just How Slim the Headset Really Is, in: ROADTOVR, 2023, https://www.roadtovr.com/meta-quest-3-teardown-ifixit-repair/, online in: https://www.roadtovr.com/ [08.02.2024].
7. Hayden, S.: Vive Ultimate Tracker Gets Beta Support for Third-Party PC VR Headsets, in: ROADTOVR, 2024, https://www.roadtovr.com/vive-ultimate-tracker-quest-index-pico/, online in: https://www.roadtovr.com/ [08.02.2024].
8. n.a.: What to Watch: February 2024 Highlights, n: Meta Quest-Blog, 2024, https://www.meta.com/de-de/blog/quest/what-to-watch-free-meta-quest-tv-vr-film, online in: https://www.meta.com/at/ [08.02.2024]

by vinzenz.weber -

XR 10 // Wrap up

As we close the chapter on a semester filled with extensive learning about Extended Reality (XR), it’s an opportune moment to reflect on the ground we’ve covered and to anticipate the exciting journey ahead. Our exploration has spanned a diverse range of topics, each offering a unique perspective on the rapidly evolving world of XR.

  1. XR 1// Evolving Interaction and Experiences in XR: I began this journey by delving into the dynamic world of XR interactions, examining how they’ve transformed over time and what the future holds.
  2. XR 2 // The Evolution and Current Landscape of Extended Reality: This post offered a comprehensive view of XR’s growth trajectory and its current state, setting the stage for this subsequent deep dives.
  3. XR 3 // VR Interactions: Controller vs Body Tracking: A detailed comparison between controller-based and body-tracking interfaces in VR, highlighting their unique advantages and potential applications.
  4. XR 4 // Enhancing Virtual Reality: The Power of Haptic Feedback: I explored the sensory dimension of VR, focusing on how haptic feedback intensifies the immersive experience.
  5. XR 5 // Interesting Case Study: HoloStudio UI and Interaction Design by Microsoft: This case study provided insights into practical applications of XR, emphasizing user interface and interaction design.
  6. XR 6 // UX in Mixed Reality: I discussed the intricacies of user experience design in the mixed reality spectrum, emphasizing its importance in creating engaging and intuitive applications.
  7. XR 7 // Dive into Vision OS Guidelines: This post was dedicated to understanding the best practices and guidelines in designing for XR platforms, particularly the Vision OS.
  8. XR 8 // Beyond Gaming: XR in the Entertainment Industry: I expanded our view to see how XR is revolutionizing the broader entertainment sector, beyond just gaming.
  9. XR 9 // XR in the Military Complex: My exploration concluded with an examination of XR’s applications in military training and strategy, showcasing its diverse utility.

What now?

Last semester’s curriculum primarily revolved around theoretical aspects. For the upcoming semester, I aim to adopt a more practical approach towards the subject. This will involve actively engaging with various XR applications and models for comparative analysis and learning. Additionally, I plan to implement the concepts learned in my own projects and create case studies for them.

by vinzenz.weber -

XR 9 // XR in the Military Complex

In this semester’s research project, I explored various facets of Extended Reality (XR). Today, let’s delve into a lesser-discussed but significant aspect: the role of XR in the military defense complex. Notably, the military sector is one of the leading investors and developers in this technology.

Training with Mixed Reality

Mixed Reality (MR), blending elements of Augmented Reality (AR) and Virtual Reality (VR), has revolutionized military training. Historically, MR applications like the Swiss tank-driving simulator from the 1970s laid the groundwork for today’s sophisticated systems.

One prominent example is the U.S. Army’s Synthetic Training Environment (STE). This advanced system merges virtual and augmented reality to simulate a wide range of scenarios, from urban warfare to counterinsurgency operations, thus providing immersive training experiences. The STE is a comprehensive platform integrating virtual, live, and collective training elements, designed to be portable and cost-effective. It includes the Reconfigurable Virtual Collective Trainer (RVCT), which offers training for various military vehicles and infantry units​​​​.

Mixed Reality in Combat

MR’s role extends beyond training to actual combat operations. It significantly enhances situational awareness by providing soldiers with real-time information through contextually relevant visualizations. This includes displaying crucial data like maps, navigation, and enemy locations seamlessly.

Soldiers training with Microsoft’s IVAS

A key development in this area is the Integrated Visual Augmentation System (IVAS), a collaborative effort between Microsoft and the U.S. Army. Based on Microsoft’s HoloLens technology, IVAS delivers advanced capabilities such as rapid target acquisition, enhanced situational awareness, and improved navigational tools. It integrates various technologies like thermal imagery, sensors, GPS, and night vision to give soldiers a comprehensive view of the battlefield. This technology is not only pivotal for training but also holds immense potential for real-world combat operations, allowing soldiers to plan and execute missions with enhanced precision and information​​​​.

Support Functions

MR’s applications in the military also extend to support functions. It can transform maintenance and repair processes by overlaying relevant instructions onto real-world objects, aiding technicians and mechanics in performing tasks more efficiently.

In medical support and telemedicine, MR can overlay digital content such as instructions and patient data, facilitating accurate and efficient medical procedures in challenging environments.

Conclusion

MR technology is a game-changer in military applications, enhancing various aspects of operations. While it offers immense benefits in training, situational awareness, and support functions, there are challenges to consider. For instance, overreliance on technology can lead to operational inefficiencies if not managed properly. The concept of „HUD-Cripple,“ prevalent among Navy aviators, highlights the risk of becoming overly dependent on technological aids to the extent that performance without them is significantly impaired.

Moreover, the use of MR in combat situations introduces ethical dilemmas around warfare conduct. The enhanced capabilities provided by MR could lead to debates about the fairness and humanitarian implications of using such advanced technology in conflicts. This necessitates a balance between technological advancement and adherence to international warfare norms and ethics.

The responsibility of XR designers and developers in this context is profound. They must not only focus on the technical and functional aspects of MR systems but also consider their broader societal implications. This includes ensuring that the technology is used responsibly and in accordance with ethical standards. Designers and developers need to collaborate closely with military experts, ethicists, and psychologists to understand the full spectrum of impacts their creations might have. Furthermore, there should be ongoing assessment and adjustment of these technologies to align with evolving ethical standards and societal values.

As we venture further into this technologically advanced era, the responsibility of XR professionals extends beyond innovation, encompassing the ethical stewardship of their creations in the complex domain of military applications.

Sources

by vinzenz.weber -

XR 8 // Beyond Gaming: XR in the Entertainment Industry

XR is a technology that has been gaining popularity in the entertainment industry. While gaming is a major part of XR, this post would explore its other applications in entertainment, such as virtual concerts, immersive theater, and interactive art installations.

Virtual Concerts: A New Stage for Artists and Fans

Imagine attending a concert by your favorite artist from the comfort of your living room, yet feeling as though you’re right there in the front row. XR makes this possible. Virtual concerts in XR are not just about streaming a live performance; they are about creating an immersive, interactive experience. Fans can choose different viewpoints, interact with the environment, and even feel the vibration of the music through haptic feedback technology.

Artists like Travis Scott and Marshmello have already experimented with these concepts, drawing millions of virtual attendees. These events aren’t just concerts; they’re hyper-realistic experiences blending music, visual art, and digital interaction.

Meta is also pushing strongly into this direction by hosting live concerts on their Meta Quest Plattform. There will be for example a Lice concert by imagine Dragon at June 15th on this plattform.

Immersive Theater: Blurring the Lines Between Audience and Performer

Theater has always been an immersive experience, but XR takes this immersion to a new level. Unlike traditional theater, where the audience is a passive observer, XR theater can make viewers a part of the performance. Through VR headsets or AR applications, audience members can experience different narratives from multiple perspectives, interact with the performers, or even influence the outcome of the story.

Companies like Punchdrunk and Magic Leap are pioneering in this space, creating experiences where the line between audience and performer is blurred, leading to a more intimate and personal form of storytelling.

Interactive Art Installations: Stepping into the Canvas

Art has always been a medium for expression and experience, but XR adds an interactive dimension to it. Interactive art installations using XR technologies allow viewers to step into the artwork, manipulate elements, and experience the art in a multi-sensory manner. This form of art is not just to be seen but to be experienced and interacted with.

Artists like Refik Anadol and teamLab are at the forefront, creating stunning visual landscapes that respond to and evolve with viewer interactions. These installations are not static; they are dynamic and alive, offering a personalized experience to every viewer.

Conclusion: A New Era of Entertainment

XR in entertainment is more than a technological advancement; it’s a paradigm shift in how we experience art, music, and storytelling. It’s about creating worlds that we can step into, interact with, and be a part of. As we look to the future, the possibilities are boundless. We’re not just witnessing a change in entertainment; we’re participating in the birth of entirely new forms of expression and experience.

This is just the beginning. As XR technologies continue to evolve, we can expect to see even more innovative and immersive experiences that challenge our perceptions of reality and entertainment. The future of entertainment is here, and it’s virtual, augmented, and mixed.

Sources

by vinzenz.weber -

XR 7 // Dive into Vision OS Guidelines

Apple is stepping into the future with its highly anticipated mixed reality headset, introducing a groundbreaking operating system: Vision OS. This isn’t just another tech release; it’s a glimpse into what could shape the future of mixed reality. Diving into Apple’s developer resources, we’re offered a preview of this innovative landscape. Interestingly, Apple steers clear of terms like „Virtual Reality“ or „Augmented Reality,“ opting for „Spatial Design.“ This isn’t just a play on words; it’s a strategic move to set Vision OS apart from other mixed reality platforms.

A Familiar Yet Revolutionary Interface

Vision OS brings a familiar feel to an entirely new dimension. The interface mirrors iPad app designs but in a dynamic 3D space. It’s not just about aesthetics; it’s about functionality. Windows in Vision OS adapt to lighting conditions, and introduce intuitive controls for movement, resizing, and closing. The system also integrates extra depth layers to establish a hierarchy between elements, all while maintaining a spacing akin to iPad apps. Apple’s strategy here is clear: use familiar paradigms and patterns to ease users into this new spatial environment.

Human-Centric Design

At its core, Vision OS is designed with a keen focus on human interaction. The view plane is usually centered and horizontally aligned, aligning with the natural human line of sight. Apple makes a notable design choice here: windows don’t follow your head movements; they’re anchored in 3D space. Familiar gestures, like pinch to zoom, are still part of the experience, offering users various ways to interact with apps.

Dimensional Depth

In Vision OS, apps are designed to be in tune with the user’s real-world surroundings. Apple emphasizes UI elements crafted from a glass-like material, allowing backgrounds to subtly shine through and create a sense of real 3D objects blending into the room. Controls and menus are thoughtfully positioned closer to the user, making them more accessible and easier to perceive. Apple’s attention to detail extends to how windows and apps interact with their environment, casting realistic shadows and emitting light. The depth usage is subtle, and the windows closer to the user are smaller, enhancing the sense of spatial awareness.

Immersive Experiences

Vision OS categorizes immersion into three levels:

  1. App window floating in front of user
  2. panoramic window wrpaping around the user
  3. Completely surround vr experience

The system smartly fades out the surrounding environment to focus on the selected window. With 3D audio enhancement, these immersive experiences are reserved for moments that truly matter, always allowing an easy return to reality.

Authenticity Is Key

In Vision OS, authenticity is paramount. Apps are expected to be engaging, immersive, and make sense for this new device. It’s not just about creating something new; it’s about creating something that feels right for the platform.

Conclusion

Vision OS is Apple’s bold statement in the mixed reality arena, blending the familiar with innovative spatial design. With its human-centric approach, dimensional depth, varying levels of immersion, and emphasis on authenticity, Vision OS is poised to revolutionize how we interact with technology. It’s more than an operating system; it’s a new way to experience the digital world.

Sources

by vinzenz.weber -

XR 6 // UX in Mixed Reality

Physical Considerations

  • Environmental Interface: Designers must consider the entire surrounding environment as a potential interface, moving beyond the confines of a flat screen.
  • Comfortable Focusing Range: Interactive elements should be placed within a range of half a meter to 20 meters, the comfortable focusing distance for human eyes.
  • Beyond Reach: For interacting with objects 20 meters away, MR utilizes tools like handheld controllers or technologies such as eye tracking and hand recognition.

Eye Movement

The human eye comfortably moves 30°-35° in all directions, creating a field of view (FoV) of about 60°. Key UI elements should be placed within this range for easy accessibility.

Key Elements are arranged in a FoV of ~60°

Arms Reach

The average arm’s length is 50–70 cm. Essential interactions should be positioned at this distance for ease of use.

Designing for Distance

Drawing from Kharis O’Connell’s “Designing for Mixed Reality”, the interaction space is divided into three layers:

  1. Interaction Plane: Core UI elements are placed within arm’s reach.
  2. Mid-Zone: For placement of virtual objects in MR.
  3. Legibility Horizon: The limit for comfortable focus and reading, approximately 20 meters. Beyond this, only images should be used.

Addressing User Fatigue

  • Ease of Exit: Always provide a straightforward method to exit or pause, like a button.
  • Save Functionality: Allow users to save progress to prevent data loss and alleviate exit anxiety.

Scaling and Interaction

  • Button Size: Ensure buttons are large enough, with a minimum size of 2 centimeters.
  • Natural Interactions: Mimic real-world interactions, like picking up a mug by its handle.

Poses and Gestures

  • Clear Instructions: Given the novelty of MR, provide explicit instructions for poses and gestures.
  • Simplicity: Use poses and gestures sparingly to avoid overwhelming users.

Feedback and Guidance

  • System Feedback: Implement feedback mechanisms like haptic feedback or color changes when interacting with virtual elements.
  • Clear Guidance: Offer concise and clear instructions, crucial in the unfamiliar terrain of MR.

Mixed Reality is not just a new technology; it’s a new way of interacting with our world. As we design for MR, we must consider the unique physical and perceptual aspects of this medium. By focusing on intuitive interactions, comfortable viewing distances, and clear instructions, we can create MR experiences that are not only engaging but also accessible and user-friendly. The future of MR is bright, and as designers and technologists, it’s our responsibility to pave the way for this exciting new era of digital interaction.

Sources

by vinzenz.weber -

XR 5 // Interesting Case Study: HoloStudio UI and Interaction Design by Microsoft

This case study from Microsoft’s HoloStudio highlights the unique challenges and innovative solutions in designing UI and interaction experiences in mixed reality. It emphasizes the importance of user comfort, non-intrusive alerts, and seamless interaction between UI elements and holograms.

For more in-depth insights and details, you can read the full case study on Microsoft’s official website.

Problem 1: Reluctance to Move in a Virtual Environment

In HoloStudio, Microsoft initially designed the Workbench as a rectangle, akin to a real-world desk. However, they noticed a behavioral pattern: users were hesitant to move around. This reluctance was attributed to a lifetime of conditioning to stay still while working at a desk or computer. To counteract this, the Workbench was redesigned into a circular shape, eliminating the notion of a ‚front‘ position. This encouraged users to move around and explore their 3D creations from all angles.

Circular Environment to encourage Users to move around.

Key Learning: Comfort for the user is paramount. Essential UI elements, for instance, could be anchored to the virtual hand, reducing the need for physical movement to access them.

Problem 2: Disruptive Modal Dialogs

In 3D environments, traditional modal dialogs can be intrusive, popping up unexpectedly and disrupting the user experience. Microsoft experimented with various solutions and finally adopted a „thought bubble“ system. This system used visual cues like pulsing tendrils to subtly direct user attention where needed in the application, avoiding the abruptness of traditional pop-ups.

The "Thought Bubble" system included pulsing tendrils which provided a sense of direction, leading users to where their attention was needed in the app.
Dialogue Window guiding User to Action

Key Learning: Alerting users in 3D environments requires more finesse. Using attention directors like spatial sound, light rays, or thought bubbles can effectively guide users without being obtrusive.

Problem 3: UI Obstruction by Other Holograms

A common challenge in mixed reality is the obstruction of UI controls by other holograms. Microsoft’s initial solution of moving UI controls closer to the user proved uncomfortable, as it created a disconnect between the control and the associated hologram. The final solution was to ‚ghost‘ the UI control at the same distance as its associated hologram, maintaining a sense of connection while ensuring visibility and accessibility.

The solution: we ghosted the UI control, which both allowed interaction with the control and made it feel connected to the hologram it was affecting.
More Accessible UI

Key Learning: Accessibility of UI controls is crucial, even when obstructed. Innovative solutions are needed to ensure users can interact with holograms and controls seamlessly in the mixed reality environment.

In the following Blog Post I will go more in-depth into the Topic UX / UI in mixed reality.

by vinzenz.weber -

XR 4 // Enhancing Virtual Reality: The Power of Haptic Feedback

Virtual Reality (VR) has transformed from a futuristic concept into a practical tool in various fields. However, the true immersion in VR is not just about what we see or hear; it’s also about what we feel. This is where haptic feedback plays a crucial role.

Prototype Glove by Meta

The Essence of Haptic Feedback

Haptic feedback refers to the use of touch or force to communicate with users in a digital environment. In VR, this technology simulates the tactile experience, making virtual interactions more realistic and engaging.

Why Haptic Feedback Matters

  1. Improved Immersion and Realism: Haptic feedback bridges the gap between virtual and real experiences. It allows users to ‚feel‘ objects and textures in VR, enhancing the sense of presence in the virtual world.
  2. Enhanced Learning and Training: In educational and training simulations, haptic feedback can significantly improve learning outcomes. For instance, medical students can practice surgeries in a VR environment, feeling the texture and resistance of virtual tissues and organs.
  3. Increased Accessibility: For individuals with visual impairments, haptic feedback opens up new possibilities in VR, allowing them to interact with and understand virtual environments through touch.

The Haptic Fidelity Framework

This study introduces the Haptic Fidelity Framework, an innovative tool designed to assess the realism of haptic feedback in VR. This framework is categorized into Sensing, Hardware, and Software, encompassing 14 criteria that collectively define the quality and realism of haptic experiences.

The study’s evaluation of 38 papers using this framework reveals a strong correlation between the Haptic Fidelity score and the perceived realism of haptic feedback. This finding underscores the framework’s effectiveness in providing a standardized method to quantify haptic feedback’s impact in VR environments.

Scatter Plot for the analyzed Papers of the study.

This framework is a game-changer for VR developers. By using the Haptic Fidelity Framework as a guideline, developers can enhance the tactile dimension of VR, leading to more realistic and engaging user experiences.

Challenges and Future Directions

While haptic technology is promising, it faces challenges like the precise alignment of virtual and real-world interactions. Future research is focused on improving the accuracy and range of sensations that can be simulated.

Sources

by alexandra suvakova -

Augmented Reality Packaging: A Digital Revolution

In the realm of consumer products, packaging has long been overlooked as a potential powerhouse for brand engagement. Often discarded as a mere vessel for the product, packaging has now become a canvas for innovation, especially with the integration of Augmented Reality (AR). In this article, we unravel the transformative impact of AR on product packaging, exploring its roles in marketing strategies, customer engagement, and the numerous advantages it brings to businesses.

A Break from Monotony

In a saturated market, where products compete for attention on crowded shelves, traditional packaging often blends into a sea of monotony. Augmented reality packaging emerges as a game-changer, disrupting the ordinary and transforming product packaging into a dynamic interface for customer interaction and storytelling.

AR and Brand Marketing Strategy

Visualizing the Product
Augmented reality opens avenues for brands to showcase their products in a visually stunning manner. By incorporating AR markers on products, such as QR codes or URLs, customers can scan and explore products in augmented reality. This provides a unique and immersive experience, allowing customers to visualize how products look and function in real-world scenarios.

Truly Interactive
Augmented reality packaging goes beyond traditional engagement methods. Brands can embed AR experiences, including videos, animations, and interactive models, creating a captivating experience for customers. For instance, a stationery company could offer an AR-enabled gaming experience on their products, encouraging repeat purchases and distinguishing the brand from competitors.

Storytelling and Promotion
Traditional packaging has limited space for conveying brand values and product information. Augmented reality packaging serves as an extended platform, enabling brands to launch AR experiences or apps that highlight their mission, values, and offer additional information about products. It facilitates cross-promotion, showcasing related products to customers based on their interests and purchases.

Advantages of AR for Packaging

Augmented reality packaging brings several advantages to businesses, revolutionizing customer experiences and marketing strategies:

3D View of Products
AR packaging generates interactive 3D representations of products, allowing customers to view them on mobile devices. This enhances understanding and engagement, providing a better grasp of the product’s features and functionality.

Interactive Product Experience
AR adds interactivity to product packaging, enabling customers to scan QR codes for immersive experiences such as video games. This not only makes packaging more enjoyable but also serves as a marketing tool, enhancing the product’s appeal.

Customized Shopping
AR allows for personalized shopping experiences, with customers scanning product boxes to access tailored product information based on their preferences. This creates a stronger connection between customers and products.

On-demand Demonstrations
Engaging product demonstrations can be created using AR, offering customers a more interesting and informative way to learn about a product. This dynamic approach enhances the learning experience and contributes to better retention.

Tracking Customer Behaviors
AR packaging can be used to track customer behavior at the point of sale. Brands can analyze how customers interact with the packaging and use this data to enhance design and optimize the effectiveness of AR applications.

Use Cases of Augmented Reality Packaging

Lego: The renowned toy company Lego utilizes augmented reality packaging with an accompanying app. Customers can scan Lego sets to see 3D representations, bringing the products to life in a virtual space.

Coca-Cola: Coca-Cola embraces augmented reality with an app that provides insights into the company’s history and includes interactive games. This innovative approach enhances brand engagement.

Nespresso: Nestle’s Nespresso uses augmented reality packaging to allow users to scan and visualize how brewed coffees will look. It serves as an educational tool, explaining the brewing process to customers.

Carrefour: The French retail company Carrefour employs augmented reality food packaging to enable customers to scan products and access nutritional information and reviews. This enhances customer decision-making.

Engage and Inform with AR Product Packaging

The integration of augmented reality into product packaging marks a revolutionary shift in the way brands engage with customers. From visualizing products in 3D to creating interactive experiences and personalized shopping, AR packaging opens new dimensions for brand communication. The examples provided, along with the myriad advantages, underscore the transformative potential of AR in packaging.

Links: