Virtual Reality (VR) has transformed from a futuristic concept into a practical tool in various fields. However, the true immersion in VR is not just about what we see or hear; it’s also about what we feel. This is where haptic feedback plays a crucial role.
Prototype Glove by Meta
The Essence of Haptic Feedback
Haptic feedback refers to the use of touch or force to communicate with users in a digital environment. In VR, this technology simulates the tactile experience, making virtual interactions more realistic and engaging.
Why Haptic Feedback Matters
Improved Immersion and Realism: Haptic feedback bridges the gap between virtual and real experiences. It allows users to ‚feel‘ objects and textures in VR, enhancing the sense of presence in the virtual world.
Enhanced Learning and Training: In educational and training simulations, haptic feedback can significantly improve learning outcomes. For instance, medical students can practice surgeries in a VR environment, feeling the texture and resistance of virtual tissues and organs.
Increased Accessibility: For individuals with visual impairments, haptic feedback opens up new possibilities in VR, allowing them to interact with and understand virtual environments through touch.
The Haptic Fidelity Framework
This study introduces the Haptic Fidelity Framework, an innovative tool designed to assess the realism of haptic feedback in VR. This framework is categorized into Sensing, Hardware, and Software, encompassing 14 criteria that collectively define the quality and realism of haptic experiences.
The study’s evaluation of 38 papers using this framework reveals a strong correlation between the Haptic Fidelity score and the perceived realism of haptic feedback. This finding underscores the framework’s effectiveness in providing a standardized method to quantify haptic feedback’s impact in VR environments.
Scatter Plot for the analyzed Papers of the study.
This framework is a game-changer for VR developers. By using the Haptic Fidelity Framework as a guideline, developers can enhance the tactile dimension of VR, leading to more realistic and engaging user experiences.
Challenges and Future Directions
While haptic technology is promising, it faces challenges like the precise alignment of virtual and real-world interactions. Future research is focused on improving the accuracy and range of sensations that can be simulated.
Virtual Reality (VR) has revolutionized the way we experience digital content, offering immersive and interactive environments. One important aspect of VR is the method of interaction, which can greatly impact the user experience and effectiveness of training applications. In this blog post, we will delve into the differences between controller-based and body tracking interactions in VR, their implications for training, and recommendations for interaction design.
The Importance of Natural Interaction
Research has shown that natural and intuitive interactions in VR enhance presence, immersion, and usability. When users can interact with virtual objects in a way that mimics real-world actions (e.G. by actually grabbing objects or by pushing Buttons with the virtual controllers), they can focus on the learning experience rather than learning how to use the equipment. This allows for a deeper level of engagement and better retention of information. As Abich and colleagues (2021) argue, training in VR is most useful when it allows for embodied, experiential interaction. The spectrum of interactions in VR ranges from „arbitrary“ interactions (e.g., double-clicking a mouse to select an object) to „natural“ interactions (e.g., rotating the hand clockwise to rotate a valve in the same direction). The more natural an interaction seems, the higher presence, immersion, and usability it affords.
Controller-Based Interactions
Controller-based interactions in VR function similarly to game console controllers. The difference lies in the tracking capabilities of VR controllers, which accurately represent the user’s hand position and movement in the virtual environment. This enables users to interact with virtual objects in a more precise manner. However, a challenge with controller-based interactions is the lack of standardized control schemes. Different VR applications may require users to interact with virtual objects using various methods, such as reaching out or using a laser pointer. This inconsistency can lead to confusion and cognitive load for users. It is important for developers to consider implementing standardized control schemes to provide a consistent and intuitive user experience across different VR applications.
Advancements in technology have enabled body tracking interactions in VR, such as eye tracking and hand/body tracking. Hand tracking, in particular, allows users to interact with virtual objects in a more natural and intuitive manner. For example, using pinch gestures to select objects or pressing virtual buttons. Hand tracking can enhance the sense of presence and immersion in VR experiences. However, similar to controller-based interactions, there is a lack of standardized gestures for hand tracking. Users may need to learn specific gestures for different applications, which can be a barrier to intuitive interaction. To address this challenge, companies like Ultra Leap are working towards establishing standards for gesture controls in VR. By introducing standardized gestures, users can intuitively know how to perform actions without the need for extensive tutorials or guesswork.
Recommendations for Interaction Design
To optimize the user experience in VR, it is crucial to provide clear and concise tutorials for both controller-based and body tracking interactions. Instead of relying on textual instructions, show users how to perform actions through visual cues and demonstrations. This „show, don’t tell“ approach can help users quickly grasp the interaction methods without the need for extensive reading or trial and error. Additionally, consider placing important buttons and menus within reach of the user to minimize the need for physical movement within the virtual environment. For example, utilizing a popup menu on the wrist of one virtual controller can provide quick access to important functions without requiring users to navigate across the virtual room. These design considerations can help reduce cognitive load and ensure a seamless and intuitive user experience in VR.
Menu solution by Ultraleap
In conclusion, the choice between controller-based and body tracking interactions in VR depends on the specific application and the desired level of immersion and naturalness. Both interaction methods have their advantages and challenges, but with thoughtful design, standardized gestures, and concise tutorials, VR can offer truly immersive and effective training experiences. As VR technology continues to evolve, it is important for developers and researchers to collaborate in establishing best practices and standards for interaction design to unlock the full potential of VR for training and other applications.
From Sensorama to Apple Vision Pro: A Journey Through XR’s History
The Beginnings: Sensorama and the First HMD
The journey of Extended Reality (XR) dates back to 1956 when cinematographer Morton Heilig created Sensorama, the first Virtual Reality (VR) machine.
This innovative movie booth combined 3D, stereoscopiccolor video with audio, smells, and a vibrating chair, immersing viewers in a unique cinematic experience. Heilig’s pioneering work didn’t stop there; in 1960, he patented the first head-mounted display (HMD), merging stereoscopic 3D images with stereo sound, laying the groundwork for future VR technologies.
Early Steps in Augmented Reality: The Sword of Damocles
By 1965, the field of XR took another significant leap with Ivan Sutherland’s development of „The Sword of Damocles“ Considered the first augmented reality (AR) HMD and tracking system, it aimed to enhance users‘ perception of the world. Despite its primitive user interface and simple wireframe graphics, it marked a crucial step in the evolution of AR.
Mediated Reality and the Reality-Virtuality Continuum
In the 1970s, Steve Mann’s research into mediated reality, which later influenced tech giants like Google, Apple, and Microsoft, focused on augmenting human perception through digital overlays in the real world. Building on this concept, Paul Milgram and Fumio Kishino introduced the Reality-Virtuality continuum in 1994, illustrating a spectrum of experiences from purely real to purely virtual environments.
The 1990s: Pioneering AR and the Birth of Sportsvision
The 1990s saw Thomas Caudell & David Mizell develop the first see-through HMD, coining the term „augmented reality.“ In a significant mainstream breakthrough, Sportsvision broadcast the first live NFL game in 1998 with a yellow yard marker overlay, revolutionizing sports broadcasting.
Modern Advancements: The 2010s Onward
The 2010s heralded rapid advancements in XR technology. Key developments included:
First Oculus Rift Prototype: A milestone in VR technology.
2014 – A Landmark Year: Sony and Samsung jumped into the VR headset market, while Google launched the affordable Cardboard VR viewer and the Google Glass AR glasses.
Microsoft’s HoloLens: Released in 2016, it introduced a more interactive AR experience, often referred to as „mixed reality.“
Pokémon GO: This 2016 game brought AR to the masses, demonstrating the technology’s mainstream appeal.
A Push into the Mainstream
Apple’s AR Kit and Google’s MR Toolkit made AR accessible on smartphones, broadening the technology’s reach. In 2017, the IKEA Place app showcased AR’s practical use in retail, allowing users to visualize furniture in their homes before purchasing.
The Current State: Meta Quest 3 and Apple Vision Pro
Today, we see state-of-the-art AR and VR combinations through devices like Meta Quest 3. The recent announcement of Apple Vision Pro signals a potential expansion in audience reach, acceptance, and continued research and development in mixed reality technologies.
Welcome to the world of Extended Reality (XR), where technology is redefining our reality. In this blog, we’re going to explore how XR, a mix of Virtual Reality (VR) and Augmented Reality (AR), is changing not just gaming and entertainment, but also making waves in fields like science, architecture, and communication.
Imagine a world where the real and digital blend seamlessly. This is what XR is bringing to life, transforming how we interact and experience the world around us. We’ll take a look at how different industries are adapting to this new era, where digital and physical experiences are intertwined.
We’ll also discuss the race to develop cutting-edge XR technology. This isn’t just about creating new gadgets; it’s about a whole new market that’s opening up, changing the way we play, learn, and connect with each other.
One key area i will focus on is the gaming and entertainment industry. XR is revolutionizing these fields, creating virtual spaces where players can interact in ways that were once only imaginable. But XR’s impact goes beyond gaming; we’ll also explore its potential in other areas, reflecting on its growth and the new opportunities it creates.
On a personal note, I’ve always been intrigued by technological advancements. My early experiences with VR games sparked a passion for being part of this evolving technology. Through this blog, I hope to share not only the exciting developments in XR but also consider its ethical implications, ensuring we keep a human-centered approach in its adoption.
@ apple
Research Questions
What are the possibilities of interactions and experiences in XR?
How did XR evolve in the past?
What is currently being used, and what can we expect in the future?
Challenges
The biggest challenge of this project is probably the rapid advancements in the industry, as it is still a niche topic with many experimental projects and features. It is uncertain which new developments will be significant breakthroughs and which ones will be failures and quickly forgotten. The direction of XR and which features will ultimately be adopted by the masses remain unclear.
Relevance
XR is shaping the future of various areas, including Entertainment, Science and Education, and Communications. Its relevance is increasing day by day and it has the potential to completely transform our understanding of how we interact with media, our environment, and each other.
XR can revolutionise storytelling and entertainment, creating interactive and immersive narratives that engage users on a deeper level.
XR can provide immersive training experiences for various fields, such as medicine, aviation, and military, allowing individuals to practice and gain skills in realistic virtual environments.
XR can enhance remote collaboration and communication, enabling people from different locations to interact and work together as if they were in the same physical space.
XR can improve accessibility by creating inclusive experiences for individuals with disabilities, allowing them to participate in activities and interactions that may be challenging in the physical world.
XR can enhance design and prototyping processes, enabling designers to visualize and iterate on concepts in three-dimensional virtual spaces before physical production.
Next Steps
In the first section, I will discuss past advancements in the XR industry and how they are used today. After that, I will delve into specific projects from different industries that I find interesting and analyze them to gain a deeper understanding of the industry’s standards for interaction in XR. This will help identify what works well and what doesn’t. I also want to provide a profound insight on what to expect in XR in the future.
