by vinzenz.weber -

XR 8 // Beyond Gaming: XR in the Entertainment Industry

XR is a technology that has been gaining popularity in the entertainment industry. While gaming is a major part of XR, this post would explore its other applications in entertainment, such as virtual concerts, immersive theater, and interactive art installations.

Virtual Concerts: A New Stage for Artists and Fans

Imagine attending a concert by your favorite artist from the comfort of your living room, yet feeling as though you’re right there in the front row. XR makes this possible. Virtual concerts in XR are not just about streaming a live performance; they are about creating an immersive, interactive experience. Fans can choose different viewpoints, interact with the environment, and even feel the vibration of the music through haptic feedback technology.

Artists like Travis Scott and Marshmello have already experimented with these concepts, drawing millions of virtual attendees. These events aren’t just concerts; they’re hyper-realistic experiences blending music, visual art, and digital interaction.

Meta is also pushing strongly into this direction by hosting live concerts on their Meta Quest Plattform. There will be for example a Lice concert by imagine Dragon at June 15th on this plattform.

Immersive Theater: Blurring the Lines Between Audience and Performer

Theater has always been an immersive experience, but XR takes this immersion to a new level. Unlike traditional theater, where the audience is a passive observer, XR theater can make viewers a part of the performance. Through VR headsets or AR applications, audience members can experience different narratives from multiple perspectives, interact with the performers, or even influence the outcome of the story.

Companies like Punchdrunk and Magic Leap are pioneering in this space, creating experiences where the line between audience and performer is blurred, leading to a more intimate and personal form of storytelling.

Interactive Art Installations: Stepping into the Canvas

Art has always been a medium for expression and experience, but XR adds an interactive dimension to it. Interactive art installations using XR technologies allow viewers to step into the artwork, manipulate elements, and experience the art in a multi-sensory manner. This form of art is not just to be seen but to be experienced and interacted with.

Artists like Refik Anadol and teamLab are at the forefront, creating stunning visual landscapes that respond to and evolve with viewer interactions. These installations are not static; they are dynamic and alive, offering a personalized experience to every viewer.

Conclusion: A New Era of Entertainment

XR in entertainment is more than a technological advancement; it’s a paradigm shift in how we experience art, music, and storytelling. It’s about creating worlds that we can step into, interact with, and be a part of. As we look to the future, the possibilities are boundless. We’re not just witnessing a change in entertainment; we’re participating in the birth of entirely new forms of expression and experience.

This is just the beginning. As XR technologies continue to evolve, we can expect to see even more innovative and immersive experiences that challenge our perceptions of reality and entertainment. The future of entertainment is here, and it’s virtual, augmented, and mixed.

Sources

by vinzenz.weber -

XR 7 // Dive into Vision OS Guidelines

Apple is stepping into the future with its highly anticipated mixed reality headset, introducing a groundbreaking operating system: Vision OS. This isn’t just another tech release; it’s a glimpse into what could shape the future of mixed reality. Diving into Apple’s developer resources, we’re offered a preview of this innovative landscape. Interestingly, Apple steers clear of terms like „Virtual Reality“ or „Augmented Reality,“ opting for „Spatial Design.“ This isn’t just a play on words; it’s a strategic move to set Vision OS apart from other mixed reality platforms.

A Familiar Yet Revolutionary Interface

Vision OS brings a familiar feel to an entirely new dimension. The interface mirrors iPad app designs but in a dynamic 3D space. It’s not just about aesthetics; it’s about functionality. Windows in Vision OS adapt to lighting conditions, and introduce intuitive controls for movement, resizing, and closing. The system also integrates extra depth layers to establish a hierarchy between elements, all while maintaining a spacing akin to iPad apps. Apple’s strategy here is clear: use familiar paradigms and patterns to ease users into this new spatial environment.

Human-Centric Design

At its core, Vision OS is designed with a keen focus on human interaction. The view plane is usually centered and horizontally aligned, aligning with the natural human line of sight. Apple makes a notable design choice here: windows don’t follow your head movements; they’re anchored in 3D space. Familiar gestures, like pinch to zoom, are still part of the experience, offering users various ways to interact with apps.

Dimensional Depth

In Vision OS, apps are designed to be in tune with the user’s real-world surroundings. Apple emphasizes UI elements crafted from a glass-like material, allowing backgrounds to subtly shine through and create a sense of real 3D objects blending into the room. Controls and menus are thoughtfully positioned closer to the user, making them more accessible and easier to perceive. Apple’s attention to detail extends to how windows and apps interact with their environment, casting realistic shadows and emitting light. The depth usage is subtle, and the windows closer to the user are smaller, enhancing the sense of spatial awareness.

Immersive Experiences

Vision OS categorizes immersion into three levels:

  1. App window floating in front of user
  2. panoramic window wrpaping around the user
  3. Completely surround vr experience

The system smartly fades out the surrounding environment to focus on the selected window. With 3D audio enhancement, these immersive experiences are reserved for moments that truly matter, always allowing an easy return to reality.

Authenticity Is Key

In Vision OS, authenticity is paramount. Apps are expected to be engaging, immersive, and make sense for this new device. It’s not just about creating something new; it’s about creating something that feels right for the platform.

Conclusion

Vision OS is Apple’s bold statement in the mixed reality arena, blending the familiar with innovative spatial design. With its human-centric approach, dimensional depth, varying levels of immersion, and emphasis on authenticity, Vision OS is poised to revolutionize how we interact with technology. It’s more than an operating system; it’s a new way to experience the digital world.

Sources

by vinzenz.weber -

XR 6 // UX in Mixed Reality

Physical Considerations

  • Environmental Interface: Designers must consider the entire surrounding environment as a potential interface, moving beyond the confines of a flat screen.
  • Comfortable Focusing Range: Interactive elements should be placed within a range of half a meter to 20 meters, the comfortable focusing distance for human eyes.
  • Beyond Reach: For interacting with objects 20 meters away, MR utilizes tools like handheld controllers or technologies such as eye tracking and hand recognition.

Eye Movement

The human eye comfortably moves 30°-35° in all directions, creating a field of view (FoV) of about 60°. Key UI elements should be placed within this range for easy accessibility.

Key Elements are arranged in a FoV of ~60°

Arms Reach

The average arm’s length is 50–70 cm. Essential interactions should be positioned at this distance for ease of use.

Designing for Distance

Drawing from Kharis O’Connell’s “Designing for Mixed Reality”, the interaction space is divided into three layers:

  1. Interaction Plane: Core UI elements are placed within arm’s reach.
  2. Mid-Zone: For placement of virtual objects in MR.
  3. Legibility Horizon: The limit for comfortable focus and reading, approximately 20 meters. Beyond this, only images should be used.

Addressing User Fatigue

  • Ease of Exit: Always provide a straightforward method to exit or pause, like a button.
  • Save Functionality: Allow users to save progress to prevent data loss and alleviate exit anxiety.

Scaling and Interaction

  • Button Size: Ensure buttons are large enough, with a minimum size of 2 centimeters.
  • Natural Interactions: Mimic real-world interactions, like picking up a mug by its handle.

Poses and Gestures

  • Clear Instructions: Given the novelty of MR, provide explicit instructions for poses and gestures.
  • Simplicity: Use poses and gestures sparingly to avoid overwhelming users.

Feedback and Guidance

  • System Feedback: Implement feedback mechanisms like haptic feedback or color changes when interacting with virtual elements.
  • Clear Guidance: Offer concise and clear instructions, crucial in the unfamiliar terrain of MR.

Mixed Reality is not just a new technology; it’s a new way of interacting with our world. As we design for MR, we must consider the unique physical and perceptual aspects of this medium. By focusing on intuitive interactions, comfortable viewing distances, and clear instructions, we can create MR experiences that are not only engaging but also accessible and user-friendly. The future of MR is bright, and as designers and technologists, it’s our responsibility to pave the way for this exciting new era of digital interaction.

Sources

by vinzenz.weber -

XR 5 // Interesting Case Study: HoloStudio UI and Interaction Design by Microsoft

This case study from Microsoft’s HoloStudio highlights the unique challenges and innovative solutions in designing UI and interaction experiences in mixed reality. It emphasizes the importance of user comfort, non-intrusive alerts, and seamless interaction between UI elements and holograms.

For more in-depth insights and details, you can read the full case study on Microsoft’s official website.

Problem 1: Reluctance to Move in a Virtual Environment

In HoloStudio, Microsoft initially designed the Workbench as a rectangle, akin to a real-world desk. However, they noticed a behavioral pattern: users were hesitant to move around. This reluctance was attributed to a lifetime of conditioning to stay still while working at a desk or computer. To counteract this, the Workbench was redesigned into a circular shape, eliminating the notion of a ‚front‘ position. This encouraged users to move around and explore their 3D creations from all angles.

Circular Environment to encourage Users to move around.

Key Learning: Comfort for the user is paramount. Essential UI elements, for instance, could be anchored to the virtual hand, reducing the need for physical movement to access them.

Problem 2: Disruptive Modal Dialogs

In 3D environments, traditional modal dialogs can be intrusive, popping up unexpectedly and disrupting the user experience. Microsoft experimented with various solutions and finally adopted a „thought bubble“ system. This system used visual cues like pulsing tendrils to subtly direct user attention where needed in the application, avoiding the abruptness of traditional pop-ups.

The "Thought Bubble" system included pulsing tendrils which provided a sense of direction, leading users to where their attention was needed in the app.
Dialogue Window guiding User to Action

Key Learning: Alerting users in 3D environments requires more finesse. Using attention directors like spatial sound, light rays, or thought bubbles can effectively guide users without being obtrusive.

Problem 3: UI Obstruction by Other Holograms

A common challenge in mixed reality is the obstruction of UI controls by other holograms. Microsoft’s initial solution of moving UI controls closer to the user proved uncomfortable, as it created a disconnect between the control and the associated hologram. The final solution was to ‚ghost‘ the UI control at the same distance as its associated hologram, maintaining a sense of connection while ensuring visibility and accessibility.

The solution: we ghosted the UI control, which both allowed interaction with the control and made it feel connected to the hologram it was affecting.
More Accessible UI

Key Learning: Accessibility of UI controls is crucial, even when obstructed. Innovative solutions are needed to ensure users can interact with holograms and controls seamlessly in the mixed reality environment.

In the following Blog Post I will go more in-depth into the Topic UX / UI in mixed reality.

by vinzenz.weber -

XR 4 // Enhancing Virtual Reality: The Power of Haptic Feedback

Virtual Reality (VR) has transformed from a futuristic concept into a practical tool in various fields. However, the true immersion in VR is not just about what we see or hear; it’s also about what we feel. This is where haptic feedback plays a crucial role.

Prototype Glove by Meta

The Essence of Haptic Feedback

Haptic feedback refers to the use of touch or force to communicate with users in a digital environment. In VR, this technology simulates the tactile experience, making virtual interactions more realistic and engaging.

Why Haptic Feedback Matters

  1. Improved Immersion and Realism: Haptic feedback bridges the gap between virtual and real experiences. It allows users to ‚feel‘ objects and textures in VR, enhancing the sense of presence in the virtual world.
  2. Enhanced Learning and Training: In educational and training simulations, haptic feedback can significantly improve learning outcomes. For instance, medical students can practice surgeries in a VR environment, feeling the texture and resistance of virtual tissues and organs.
  3. Increased Accessibility: For individuals with visual impairments, haptic feedback opens up new possibilities in VR, allowing them to interact with and understand virtual environments through touch.

The Haptic Fidelity Framework

This study introduces the Haptic Fidelity Framework, an innovative tool designed to assess the realism of haptic feedback in VR. This framework is categorized into Sensing, Hardware, and Software, encompassing 14 criteria that collectively define the quality and realism of haptic experiences.

The study’s evaluation of 38 papers using this framework reveals a strong correlation between the Haptic Fidelity score and the perceived realism of haptic feedback. This finding underscores the framework’s effectiveness in providing a standardized method to quantify haptic feedback’s impact in VR environments.

Scatter Plot for the analyzed Papers of the study.

This framework is a game-changer for VR developers. By using the Haptic Fidelity Framework as a guideline, developers can enhance the tactile dimension of VR, leading to more realistic and engaging user experiences.

Challenges and Future Directions

While haptic technology is promising, it faces challenges like the precise alignment of virtual and real-world interactions. Future research is focused on improving the accuracy and range of sensations that can be simulated.

Sources

by helene.goedl -

03 | Digital Fashion – Topic Overview

Digital fashion refers to clothing and accessories that exist purely in the digital realm, created using computer-generated imagery (CGI) and 3D modelling techniques. It combines elements of fashion, technology, and art to create virtual garments that can be worn and showcased in digital spaces, such as virtual reality (VR) environments, online gaming platforms, and social media. Because of these factors, digital fashion holds great potential for an interdisciplinary case study in the field of media design. It combines factors from different disciplines and intersects in a way that presents both enormous potential and considerable challenges in the design process. Over the next blog entries, I will present several factors of digital fashion that are relevant and interesting to investigate in a further research project. This blog entry presents a brief topic overview to establish some of the main areas of the topic.

History

As an introductory entry, it will be interesting to take a closer look at the history of digital fashion. Arguably, the core concept of digital fashion has existed since the development of digital animation and computer games. These two areas have provided the most opportunities to showcase digitally designed and created garments. In a future blog entry, the history of digital fashion will be revisited in more detail. 

Digital Fashion for Avatars 

In the gaming and virtual reality space, digital fashion is gaining popularity as a way for users to customize the appearance of their avatars. Players can purchase or earn virtual clothing items to enhance the aesthetics of their digital personas.

Collaborations and Digital Fashion Brands

Traditional fashion brands are starting to collaborate with digital artists and designers to create limited-edition virtual collections. Some brands are also emerging specifically as digital fashion entities, focusing on creating unique and innovative digital clothing. Some of the collaboration projects also tie in with the previous topic as they have taken place in the video game sphere. Most notably, the 2021 collaboration between luxury fashion brand Balenciaga and the gaming studio Epic Games for the popular game Fortnite has made waves in the business. A future blog entry will explore said collaboration in more detail as a case study.

Virtual Fashion Shows

Designers are increasingly exploring the possibilities of presenting their collections through virtual fashion shows. These events take place in digital environments, allowing for unique and immersive presentations that go beyond the constraints of traditional runway shows. This has gained relevance especially in the pandemic years and will be looked at in more detail, with a special focus on differences in potential between real life and virtual fashion shows and what the introduction of the new possibilities could mean for the fashion industry in the future.

Sustainability

Digital fashion is often touted as a more sustainable alternative to traditional fashion. Since there is no physical production involved, there is no need for raw materials, manufacturing, or shipping, potentially reducing the environmental impact associated with the fashion industry. At the same time, the discussion around digital products and their sustainability is a point of contention on its own, since the ever growing need for larger server farms, computers and their energy consumption need to be weighed against traditional forms of production. 

NFTs and Digital Ownership

Some digital fashion items are being sold as NFTs, providing a sense of ownership and authenticity in the digital space. Buyers can use these NFTs to showcase and trade their virtual fashion items within specific platforms. This is especially interesting because it brings up discussions around the value of digital products in general, the merit of owning a purely virtual item and the relationship to the analogue equivalent of attributing value to fashion items. An intersting point of discussion for this topic is also the relationship between original fashion pieces and fake products or counterfeit items in the real world and in digital space.

Augmented Reality (AR) Fashion

AR technology allows users to superimpose digital clothing onto their real-world surroundings through smartphone apps or AR glasses. This enables users to experiment with different looks without physically trying on clothes. This also ties in with the previous topic of digital ownership and how this might be handled in AR spaces.

3D Modelling for Fashion Design

Digital fashion designers is mainly done in 3D modelling software to create detailed and realistic virtual garments. This process enables them to experiment with unconventional materials, shapes, and designs that may be challenging or impossible to produce in the physical world. A closer exploration of material qualities and possibilities in digital space will be continued in its own blog post. 

Digital fashion is a dynamic and evolving field that continues to push the boundaries of creativity and technology. As technology advances, we can expect to see even more integration of digital fashion into our virtual and augmented reality experiences.

by david.lasslberger -

#09 Ethical considerations regarding VR and modern healthcare technology

Many times during my research, I have come across the importance of ethical considerations for the use of VR (both as a consumer product and a medical instrument) and about the usage of modern technology in healthcare in general. In this blog post, I want to delve deeper into some of the challenges and important lessons regarding ethics in this environment.

Using VR and modern digital technology in treatment settings introduces a host of ethical considerations. One of the primary concerns revolves around patient privacy and data security. With the collection and utilization of sensitive health information through digital platforms, ensuring robust safeguards against data breaches and unauthorized access becomes paramount. Health information is sensitive information. When protection is not set up correctly, many people can be compromised, as can be seen in [1], which is a list of the 14 biggest healthcare data breaches

Another ethical consideration is about informed consent and patient autonomy. As patients engage in VR therapy or digital therapeutics, they must be fully informed about the nature of the treatment, its potential risks and benefits, and any alternatives available. Furthermore, ensuring that patients have the capacity to provide informed consent, especially in cases involving vulnerable populations or individuals with diminished decision-making capacity, is essential. Especially regarding new technologies, patients may not know about potential risks/benefits and may be ill-informed – the goal of the professional has to be to inform the patient about these factors.

Furthermore, the equitable access to VR therapy and digital interventions raises concerns about healthcare disparities. As these technologies become more prevalent in treatment settings, ensuring equitable access across diverse socioeconomic and demographic groups becomes imperative to prevent exacerbating existing healthcare disparities. Another aspect to keep in mind are the biases of the professionals creating these services, as talked about in [2] or written about in [3] and [4]. A diverse team of people with different backgrounds can ensure that all facets are addressed and that no group of people is disadvantaged in any way.

[2] An interesting discussion about the ethics of VR and its usage as a consumer and medical product.

Addressing ethical challenges requires a multifaceted approach that encompasses technological, regulatory, and ethical considerations. Implementing robust data protection measures, such as encryption protocols and secure storage systems, can help safeguard patient privacy and mitigate the risk of data breaches.

Furthermore, integrating informed consent processes into VR therapy and digital treatment platforms empowers patients to make informed decisions about their healthcare journey. This involves providing comprehensive information about the treatment modality, including its potential risks, benefits, and alternatives, in a clear and accessible manner.

Promoting transparency and accountability within the healthcare ecosystem also plays a crucial role in addressing ethical concerns. Healthcare providers and technology developers must adhere to ethical guidelines and standards of practice, ensuring that patient interests are prioritized throughout the treatment process.

Summary & conclusion

As we navigate the evolving landscape of healthcare technology, the ethical considerations surrounding VR therapy and digital interventions will continue to shape the future of patient care. As designers, embracing a culture of ethical reflection and continuous improvement is essential to foster trust and integrity within the broader community.

Looking ahead, advancements in artificial intelligence, machine learning, and immersive technologies are most likely about to revolutionize healthcare delivery and patient outcomes. However, as we embrace these innovations, it is imperative to remain vigilant in safeguarding patient rights, promoting equitable access, and upholding ethical standards. In conclusion, navigating the ethical considerations associated with VR therapy and modern digital technology in healthcare requires a collaborative effort grounded in principles of transparency, accountability, and patient-centered care.

References used in this article and for research

The notes under the links are mostly for myself, in order to have an organised overview over the content of each source.

[1] 14 Biggest Healthcare Data Breaches [Updated 2024] | UpGuard

(22) Ethical Considerations for Virtual Reality and Immersive Experiences | LinkedIn

Ethical Challenges of Using Virtual Environments in the Assessment and Treatment of Psychopathological Disorders – PMC (nih.gov)

Ethical Considerations in Implementing VR and AR in Patient Interaction – Soft Skills for Healthcare (esoftskills.com)

Ethical Considerations in the Usage of VR/AR in Medical Technology (skill-lync.com)

Considerations and practical protocols for using virtual reality in psychological research and practice, as evidenced through exposure-based therapy | Behavior Research Methods (springer.com)

[3] Developer Bias – MikeSchinkel.com

[4] Biases in design | Bootcamp (uxdesign.cc)

by david.lasslberger -

#06 Immersion Therapy – Part 2.

This blog post is a continuation of the previous post #05, where I talked about the definition and explanation of various terms surrounding the topic of immersion therapy. In this second part, I want to go deeper into the advantages and disadvantages and specific use cases for these treatment methods. This will hopefully deepen my understanding of this subject and which elements are or are not important.

Advantages and disadvantages of this treatment method

As mentioned in previous blog posts, this type of therapy can be used to treat both physical and mental health problems.
As a side note, I found that the internet provides many resources when it comes to the treatment of mental health problems but less so for physical problems.
In the upcoming section, I will talk about advantages and disadvantages for both mental and physical problems, as these often times overlap and apply to both.

Let us discuss briefly what the distinct advantages of this digitized approach of therapy is, over more traditional ways.

  • It is a very safe and controlled environment. People can be gradually exposed to their fears/problems/anxieties. The therapist can fully control the level of exposure and also monitoring devices, e.g. for the heart rate, may be utilized to gather information. Furthermore, the patient can rest assured that they can end or pause the treatment whenever they feel overwhelmed.
  • Be more engaging and interactive experience. For example, compared to traditional talking treatments, the patient feel more immersed and connected to the experience. This helps with the efficiency of the treatment and can also be useful when dealing with patients with a low attention span or motivation, such as with children. Furthermore, VR is shown to be able to reduce the pain levels experienced by patients. It can help to break or alleviate a negative feedback loop caused by pain, dizziness and fatigue.
  • It can be time- and cost-efficient. A VR setup can be quite inexpensive and certain scenarios may be reused multiple times. Also, the treatment can easily be done in a simple room. Comparing this to e.g. treating the fear of flying where one would need to somehow simulate a flight or go flying, the ease of use becomes apparent.
  • The results produced by this method seem to be long-lasting and very positive according to several studies done. However, even though this sound positive, more research is needed. Most research done thus far is on phobias and PTSD, its use for other problems still needs to be thoroughly researched.
  • Treatment may be administered remotely. Some patients may not be able to leave their homes. VR therapy could be used to deliver training/therapy materials to patients anywhere.

Of course this type of treatment method also has disadvantages and problems, such as:

  • Technical aspects can prove difficult. Such as the initial setup cost, bugs, hardware problems and also the need to create programs/levels/scenarios for the specific treatment. This takes time and money.
  • Therapists need to be trained to utilize this tool. The adoption of VR technology as a therapy method is slow, and many mental health professionals are also not trained to work with it. This is also a problem here in Austria, where very few places (hospitals, etc.) actually work with and support it.
  • Patients may not feel open to this treatment approach and not accept it. Instead seeing it a simple gimmick with no real usage.

Let us talk very quickly about why exposure therapy works. There are several ways how it functions and creates positive effects:

  • Habituation – meaning that through exposure, people can become accustomed to their fears and decrease their reaction to the object/situations which create the fear.
  • Extinction – similar to habituation. Previous (negative) associations with objects/situations can be weakened or even replaced with positive emotions through exposure.
  • Self-efficacy – patients learn that they are strong enough to handle their fears on their own and confront them forthright.
  • Emotional processing – describes the process in which patients change their beliefs about objects/situations/activities/… and attach new, more realistic beliefs and emotions to them.
  • VR can be used to target areas in the brain responsible for pain processing and help regulate them and reduce the pain felt. For this reason, it can be used to treat several forms of chronic pains.

Summary & conclusion

Again, since this blog post is getting kind of long, I will continue and finish this subject in blog post 7 – part 3. There I will go into further detail regarding specific use cases for immersion therapy.

In this blog post, advantages and disadvantages were discussed. It is important to look at both sides and have a realistic view about this new form of therapy to be able to best use it. Furthermore, a brief overview over why this form of therapy works has been given.

References used in this article and for research

The notes under the links are mostly for myself, in order to have an organised overview over the content of each source.

Immersion Therapy for Treatment and Support of Mental Health conditions (simulationmagazine.com)
short general article about immersion therapy

What Is Virtual Reality Exposure Therapy? (choosingtherapy.com)
a more in-depth read about (VR) exposure therapy

Immersion Therapy For Anxiety: How It Works And Techniques (mantracare.org)
a more in-depth read about immersion therapy

Immersion Therapy vs. Exposure Therapy – Healthy Minded
short overview of the difference between (VR)ET and IT

What Is Exposure Therapy? (apa.org)
good, structured overview over ET

Overview – Cognitive behavioural therapy (CBT) – NHS (www.nhs.uk)
good, rather detailed read about CBT

by leonie.dunke -

8 | Theatre in the digital time

Project: Hybrid-real stage spaces

This project looks at how real and virtual spaces can work together in performances, with a focus on how audiences engage with musicians or performers. Through the creation of the VR performance „Spatial Encounters,“ they are exploring how blending physical and digital spaces can be a powerful tool for designing unique and immersive experiences. Additionally, their goal is to understand how artists collaborate in performances, specifically in the realm of musical experiences.

This special space combines real and digital elements to explore conversations between music, people, and surroundings, breaking away from traditional concerts. Participants can immerse themselves in the music without having a traditional front-facing setup. It aims to create new interactive dialogues between sound and visuals, inviting participants to focus on a unique way of listening.

Various virtual landscapes generated with VFX form the basic setup. Musicians and up to nine participants can interact simultaneously in this space. Visitors interact with their own avatars or others, influencing the virtual space. At the same time, the musical composition by the artists affects the virtual space, creating a mutual interplay between musicians and participants that constantly evolves.

The main focus of the project was on the questions:

  • How do users behave alone or in a community in different contexts, media, and formats?
  • How much influence do we allow users, and how much interaction is possible and necessary?
  • How can we transform passive viewers into active creators?

Special attention was given to ensuring a successful performance by emphasising spatial safety. Overlaps must exist in both virtual and real spaces, clearly indicating that one can move safely in the real space.

Project: Augmented reality as a communication tool

The project aims to address communication challenges in theater refurbishments and cultural venues by leveraging XR (extended reality) technologies. Traditional tools like drawings and models have limitations in conveying the complexities of spatial situations in theaters. Collaborating with experts, the project explores scenarios for renovation planning and discussions on stage productions, aiming to answer questions about presenting architectural transformations, integrating augmented reality into physical scale models and drawings, and digitally supporting planning processes using XR technology for seamless communication among stakeholders.

Basis:

Complex construction projects, including new builds and renovations of theaters and cultural venues, pose significant communication challenges. Traditional tools in architectural visualization, such as 3D views and animations, often fall short in conveying the overall spatial context. While physical models aid spatial understanding, they may lack flexibility. The need for interactivity and flexibility highlights the demand for improved communication methods.

Key Questions:

To what extent can digital technologies like Augmented Reality support communication by adding digital layers to built-scale models, drawings, and physical spaces, presenting complex architectural transformation processes comprehensibly?

Task:

In the refurbishment of the Luisenburg Festival’s natural stage, digital tools were employed to enhance the communication of the complex system structure of cavities, corridors, and stairs in the planning process. The goal was to involve all stakeholders in the decision-making process and convey the highly complex planning process in an understandable way for prompt decision-making.

Implementation:

The AR application was developed using terrestrial laser scans, drone-based photogrammetry, and CAD planning. Users can present three-dimensional content on a floor plan or physical model using a tablet, smartphone, or AR glasses. Physical objects serve as anchor points for digital content prepared in advance. Aspects such as object positioning, scaling, and user interface were considered.

Technical Tools:

WebXR Editor and Viewer

Advantages:

  • Cost savings
  • Improved planning capabilities
  • Location and time-independent planning
  • Consistent presentation regardless of weather conditions and changes in natural conditions due to snow
  • AR as an effective presentation tool for tables, models, or sketches to enhance communication

Opportunities for Digital Tools:

  • Workshop drawings and planning sketches are 2D, but a 3D blueprint is not necessarily required
  • 3D provides advantages in lighting and simulating the sun’s position without waiting for the corresponding time of day
  • Need for professional exchange between festival locations and various stakeholders at the national and international levels.

Ressources

DTHG: Abschluss-Publikation des Forschungsprojektes „Im/material Theatre Spaces“

by david.lasslberger -

#07 Immersion Therapy – Part 3.

This is part 3 of my research into the topic of immersion therapy. In this blog post, I want to explore specific use cases for this type of digital therapy.

Use cases for immersion therapy

Uses cases for VR therapy are varied. In general, a distinction between physical and psychological health problems may be considered, as these may require distinctly different treatment methods. However, it should also be noted, that physical and mental problems are often interlinked (i.e. one may develop mental health problems because of existing physical problems) and overlap frequently.
Until now, I have talked a lot about exposure therapy, which is used to treat mental health issues. However, physical problems, or the mental health problems which may arise from said health issues, may also be treated using CBT based methods. In an interview with Liana Fraenkel (MD, MPH), it was stated that Cognitive Behavioral Therapy (CBT) is as effective as other non-pharmacologic approaches such as exercise. CBT can be used to address many mental health problems, which often occur due to physical problems. Therefore, its value in physical therapy must not be understated. CBT can help treat depression, insomnia, fatigue management, and self-management, which are all symptoms of physical problems. Additionally, the techniques learned during therapy sessions can be utilized at any time, making them useful beyond the therapy sessions.

In the following section, I want to give an overview over possible uses cases. The list is roughly based from mental health problems to physical health problems:

  • Depression
  • Anxiety disorders
    e.g. social anxiety
  • Borderline personality disorder
  • Eating disorders
  • Obsessive compulsive disorder (OCD)
  • Panic disorder
  • Various different kinds of phobias
    e.g. fear of flying, animals, insects, heights, …
    Patients can be gradually exposed to stronger, more realistic depictions of their phobia
  • Post-traumatic stress disorder (PTSD)
    It is commonly used for the treatment of war veterans who suffer from PTSD. VR therapy allows them to immerse themselves in their memories and experiences in a controlled way. This can help them cope with their emotions and memories. The treatment of PTSD is an important way of helping people and preventing suicides.
  • Psychosis
  • Schizophrenia
  • Sleep problems such as insomnia
  • Alcoholism
  • Irritable bowel syndrome (IBS)
  • Chronic fatigue syndrome (CFS)
  • Fibromyalgia
  • (Partial) paralysis
    This one I know of from personal experience. As previously mentioned, my brother is partly paralyzed in his left side, especially when it comes to vision.
    One exercise was a car driving simulator using a VR headset. He had to simultaneously perform the driving actions, keep an overview over the traffic, and also deal with his trauma from his car accident.
    In another exercise, the goal was to find certain objects which were hidden in a virtual room and connect two objects with a wire.
    The goal of another training exercise was to dodge and block incoming balls which were coming toward the player.
  • Pain relief therapy / Chronic pain (such as arthritis)
    One example I came across described using VR to deal with phantom pains after the patient had lost a limb. Through the sensory information and the (movement) input, the brain is triggered and sends and receives signals to/from the still existing extremities. The setup may look like as follows:
    The patient see a virtual avatar of themselves with all limbs still attached. Over certain signals/impulses, the patients can move their amputated limbs again. This can be a highly emotional experience, but it helps over 50 % of patients to reduce pains in their daily lives.
  • Stroke
  • Cerebral Palsy 
  • Parkinson’s disease 
  • Multiple Sclerosis 
  • Especially for children, it can help them deal with ADHD, autism or developmental delays by supporting them in a fun and gamified way.
  • and many more…

Summary & conclusion

In this blog post, we took a look at specific use cases for both mental and physical health problems and how the treatment may look like. Physical and mental health problems/conditions are often interlinked and occur together – therefore, a holistic approach should be considered to increase the effectiveness of the treatment method.

References used in this article and for research

The notes under the links are mostly for myself, in order to have an organised overview over the content of each source.

Immersion Therapy for Treatment and Support of Mental Health conditions (simulationmagazine.com)
short general article about immersion therapy

What Is Virtual Reality Exposure Therapy? (choosingtherapy.com)
a more in-depth read about (VR) exposure therapy

Immersion Therapy For Anxiety: How It Works And Techniques (mantracare.org)
a more in-depth read about immersion therapy

Immersion Therapy vs. Exposure Therapy – Healthy Minded
short overview of the difference between (VR)ET and IT

What Is Exposure Therapy? (apa.org)
good, structured overview over ET

Overview – Cognitive behavioural therapy (CBT) – NHS (www.nhs.uk)
good, rather detailed read about CBT

Virtual Reality in der Schmerztherapie: Einsatz, Erfahrungen und Potenziale (onlinesicherheit.gv.at)