by leonie.dunke - 21. Dezember 202330. Dezember 2023

7 | Theatre in the digital time

Project: Virtual teaching and learning spaces

The project focuses on the development of digital, interactive training modules within a project unit. Innovative teaching and learning tools have been created for virtual reality. Virtual Reality can significantly contribute to practical learning and the acquisition of experiential knowledge, especially in spatial learning, technical preparation, and avoiding injury risks in a simulated work environment.

The use of a virtual learning space provides the opportunity to make hidden structures visible and facilitates the learning of extensive technical skills in a straightforward manner. Often, the size of learning groups and the temporal constraints tied to a specific location hinder individual experimentation and learning. The listed projects, including the 3D reconstruction of a chain hoist as a construction project for rehearsal stages and the skills training in microphone techniques through the „Sound Space“ contribution, address various aspects.

„Virtual Chain Hoist Exercise“: A VR prototype designed for a theater lab, this exercise involves selecting the right chain hoist, understanding components, and solving context-specific challenges set by a lecturer.
„Safety Signs“: Users memorize safety sign functions and engage in a fire-fighting exercise, choosing the correct fire extinguisher and maintaining the proper safety distance in this virtual learning space.
„Rehearsal Stage Setup“: This virtual space reduces rehearsal stage construction time by shifting planning to the virtual realm, addressing challenges in transforming stage designs into plans using standard materials.
„Sound Space – Microphonation“: Users explore instrument microphone setups in a virtual learning environment, comparing sound qualities in a stage and recording studio, understanding pick-up positions, and microphone types.
„Hazard Detector“: A virtual learning space in a trade fair hall, users test theoretical knowledge with a suspended rig, conducting safety checks and identifying hazards, simulating challenges faced by event technicians.
„Bending Beam“: Using theater objects, this virtual learning space teaches technical mechanics, allowing users to understand the interplay of forces in a simulated trade fair hall, providing real-time feedback and serving as a tool for theoretical teaching.
„Cybertheater 2077“: A learning game immersing users in various technical professions and theater departments, modeled after a pre-university internship, engaging prospective and advanced students, alumni, and professionals in the virtual theater world.
„Let There Be Light“: This VR project teaches lighting technology through exercises in stage lighting, object tracking, and additive color mixing. Divided into theory and practice rooms, it targets students, trainees, and the general public, with flexible upgrades based on specific learning groups or audiences.

For virtual learning to function seamlessly, strong engagement from all participants and flexible adaptation to constantly changing conditions are crucial.

Project: Digital twins of theatre technology history

„CANON“ brings together educators and students from nine European universities to create an open database and teaching approaches focused on preserving the cultural heritage of international theatre technology. The foundation of the project encounters a challenge in teaching the history of theatre technology, particularly due to the complexity of the machineries.

The project’s prototype is crafted for a teaching setting featuring a timeline on a table or wall, allowing the placement of content linked to a media database. This database’s 3D objects can be dynamically visualized in augmented reality, incorporating photos, videos, texts, or diagrams.

The key advantage lies in the ability to visually demonstrate machine functions, utilizing a dynamic 3D representation with sound, video, images, and supporting texts for easy access to theatre history. Integrated into lessons through QR codes and the CANON database, this tool supports diverse teaching scenarios (frontal, teamwork, free work, presentation) with different methods and didactics, including a „guided tour,“ the „explorer“ discovery mode, or the „flaneur style.“

Project: Bringing theatre heritage to life

„Virtual Reality Time Travel Berlin 1927“ is an immersive project that takes you on a journey to explore the history of the theatre. By integrating historical artifacts from Stadtmuseum Berlin and other archives, the project addresses the challenge of making cultural heritage accessible. It also examines how digital tools in archives and virtual reality can bring historical theatre architecture to life. The project, marking the 100th anniversary of the Große Schauspielhaus, offers both an exhibition installation and a freely accessible VR application, providing a unique and spatial experience of the theatre’s history, architecture, and art. The participants are accompanied by one of three virtual characters, each offering a different perspective of the theatre and providing a dynamic exploration the heritage.

Ressources

DTHG: Abschluss-Publikation des Forschungsprojektes „Im/material Theatre Spaces“

by leonie.dunke - 16. Dezember 202330. Dezember 2023

6 | Theatre in the digital time

Im/material theatre spaces

In my last post, I presented a work from the project Im/material theatre spaces, which offers a potential answer to my question about a digital construction rehearsal.

As the project m/material theatre spaces encompasses further works that delve into digital aspects, particularly the theme of virtual and augmented reality in the theater environment, I would like to discuss additional projects as they can serve as inspiration for my own thoughts.

This research project explores the synergy between immersive technologies and centuries-old theater knowledge. They posit that theater and virtual/augmented reality (VR/AR) share spatial immersion and methods, addressing questions of participation and changing perspectives. VR, through complete immersion, opens up new storytelling possibilities, allowing shifts in perspective and embodiment of different roles. On the other hand, AR enriches reality by overlaying it with digital content, creating a fusion between the real and digital worlds.

Research Questions

The project addresses key questions to unlock the potential of VR/AR in theater, exploring practical applications in architecture, stage design, and theater technology. Specific inquiries include the use of augmented reality in planning theater renovations, improving safety standards backstage with digital technologies, and employing immersive technologies to provide innovative access to cultural heritage.

Goals

The overarching goal of the research project is to establish theaters and event venues as ongoing hubs of technical innovation. By investigating the intersection of analogue and digital worlds, the project aims to make these new technical spaces usable for theater practitioners. The focus lies on developing prototypical solutions, communicating findings to the theater landscape, and fostering a sustained dialogue through workshops, lectures, and blog posts. The publication serves as a comprehensive overview of the project’s results, methods, and an exploration of potential future developments in the theater and cultural landscape.

Project: Digital Mounting Systems

Background:

The project addresses the lack of knowledge for assembling and dismantling complex equipment in the events industry. Not everything can be adequately conveyed through training, and many assembly and operating instructions are often impractical or too vague in paper form. To ensure safety during construction, this project aims to develop digital support.

A functional prototype, specifically addressing the AR-supported setup of a curtain rail, has been developed, serving as a practical foundation for further discussions. Through an interactive website, assembly instructions are displayed in detailed steps, supported by 3D animations. The technology allows usage on conventional screens or immersively through Augmented Reality glasses or AR functions on smartphones. The website offers flexibility for future instruction updates without the need for end-device updates.

Feedback:

The digital assembly aid was generally deemed helpful, especially due to the detailed representation of complex steps. Realistic representation was considered necessary, particularly for quick component identification. Usage on a tablet or touchscreen was preferred, while Augmented Reality glasses were viewed as promising for the future. The desire for a personal account was expressed to customize existing instructions. The application could be used for notes and specific solutions within the house or for different productions. It was noted that the application could be useful for additional instructions and the visualization of theater projects. A technical obstacle lies in providing and maintaining high-quality 3D data.

Additional Areas for Digitization in Internal Processes:

Inventory control systems
Calculation tools
Warehousing
CRM systems for customer service

How could the project be continued, and what future applications could arise from the initial prototypes? Based on project feedback, the development of an individualized and fully automated creation of stage mounting systems could be pursued. A website could be created, allowing free configuration of a modular rail system. The individualized system could then serve as the basis for the automatic generation of precise assembly instructions.

Ressources

DTHG: Abschluss-Publikation des Forschungsprojektes „Im/material Theatre Spaces“

by david.lasslberger - 10. Dezember 202311. Dezember 2023

#05 Immersion Therapy – Part 1.

In my last blog post I came across some articles talking about „Immersion Therapy“. I found this topic interesting and even though I know the basics I have yet to delve deeper into this field of therapy.
So let us do just that!

Definition of terms

Firstly, let us begin with a brief definition, which will be further expanded on in more detail in the next section. There are a bunch of terms and definitions floating around this topic. The most commonly used terms in this discourse are:

Cognitive behavioural therapy (CBT)
Exposure therapy
Virtual reality exposure therapy (VRET)
Immersion therapy

Exposure therapy describes a specific form of cognitive behavioral therapy. Immersion therapy is a more intense form of exposure therapy, where the patient is especially immersed. This can be achieved through various methods. Virtual reality exposure therapy simply describes exposure therapy which utilizes some sort of VR setup. VRET can be a part of immersion therapy, but doesn’t have to be.

In the context of this blog post, I will use the terms immersion therapy and VRET interchangeably, always meaning a form of exposure therapy which uses a VR setup to immerse the patient.

Deeper research into this topic

Next up I want to explain the four terms from above further and in more detail. The use cases for these methods will be assessed in the section „Use cases for immersion therapy“ in the next blog post #06.
I believe a strict destination between these segments helps with clarity and readability regarding this vast topic.

Firstly, let us start with cognitive behavioral therapy, since this is the building block on which all further methods are based upon. As seen in graphic 1. CBT is based on the assumption that one’s thoughts, behaviors and emotions are interlinked and can thus create a positive but also negative cycle / feedback loop. The goal of CBT is to deal with negative emotions by breaking them down into smaller parts. These can then be more easily understood and techniques can be learned how to deal with them, i.e. see them one’s problems and emotions in a new light. At the end, the patient should have a better quality of life by learning how to deal with their negative emotions. CBT is a talking treatment where patients talk to their therapist, who then can give advice on how to tackle their problems. In general, it can be said, that different from other talking methods, CBT focuses on current problems rather than searching for problems in one’s past.

Exposure therapy then builds on CBT. The goal is to help people overcome their fears and anxieties, or rather to reduce the negative reactions of people to fears and specific situations. This is done through exposure, therefore the name exposure therapy. The idea, it is predicated upon, is that people can grow stronger against their fears by willingly confronting them. In exposure therapy, patients learn how to address, process and confront their fears. Together with their therapist, patients walk through their problems, then (in a very controlled manner) expose themselves to these problems and then once again talk about their experience. This is repeated as many times necessary.

Virtual reality exposure therapy, as the name implies, is the same treatment form as exposure therapy. The only difference being, that in VRET a virtual reality setup of some kind is used. This brings some unique challenges but also advantages with it. Patients can feel way more immersed and the level of exposure can be easily controlled by the therapist. Also, patients know that they can rely on the medical specialist to turn off the VR program, should they uncomfortable or not yet ready to deal with this level of exposure.

As mentioned above, immersion therapy is a derivative of exposure therapy, which is more intense and immersive than regular exposure therapy. The patient is immersed in their traumatic experience or thought process. This is done for an extended period of time. The goal is to once again strengthen patients by providing a safe space to face their fears and problems – which ultimately leads to a better quality of life. This immersion can be done through various methods such as:

In vivo exposure
… facing the source of fear/anxiety in real life.
imaginal exposure
… trying to actively and vividly imagine the source of fear/anxiety.
Virtual reality exposure
… using a virtual reality device and setup of some kind.
Introceptive exposure
… exposing oneself to similar physical sensations which are similar to these which happen when facing the fear/anxiety. E.g. running in place to increase heart rate, similar to the sensation one with panic disorder might have, therefore training the mind to see this sensation as harmless.

Summary & conclusion

Since this topic is too big for the scope of one blog post, the advantages and disadvantages, as well as the specific use cases for this treatment method will be covered in the next upcoming blog post #06.

In this blog post I delved into the four topics of cognitive behavioral therapy (CBT), exposure therapy, immersion therapy and virtual reality exposure therapy (VRET). Both a definition and explanation of these terms were provided. Since I am not a health student, this was an unfamiliar topic and a new experience. These are hugely important fields of research and fascinating but also vast and daunting. However, I look forward to researching this topic further in the upcoming blog posts.

References used in this article and for research

Immersion Therapy for Treatment and Support of Mental Health conditions (simulationmagazine.com)

What Is Virtual Reality Exposure Therapy? (choosingtherapy.com)

Immersion Therapy For Anxiety: How It Works And Techniques (mantracare.org)

Immersion Therapy vs. Exposure Therapy – Healthy Minded

What Is Exposure Therapy? (apa.org)

Overview – Cognitive behavioural therapy (CBT) – NHS (www.nhs.uk)

image CBT

by leonie.dunke - 7. Dezember 202330. Dezember 2023

5 | Theatre in the digital time

How can new digital technology support sustainable stage set production?

I have identified model building and mock-ups as focal points, as new technologies such as VR and AR find a suitable application, particularly in space planning. They allow for the visualization of project ideas in actual size, enabling movement within the space, direct adjustments to the stage set, and evaluation of the impact of individual elements without the need for large physical models. I am exploring the potential of virtual spatial extensions for stage set planning and will also delve into existing projects that have developed concepts in this area.

Virtual modeling for stage sets offers diverse potential and brings numerous advantages:

Efficient Conceptualization: Digital modeling enables precise and efficient conceptualization of stage sets. Creative ideas can be visualized and adjusted quickly.
Collaboration and Teamwork: Virtual models promote improved collaboration across the entire production team. All stakeholders, from the set designer to the director and lighting designers, can work on the same digital model in real-time and provide feedback.
Resource Efficiency: Avoiding physical models significantly reduces material consumption, contributing not only to cost savings but also to environmental friendliness.
Sustainability: Digital models allow for a more sustainable approach. By avoiding physical materials and using eco-friendly software solutions, the ecological footprint can be minimized.
Flexibility and Adaptation: Digital models are easily adaptable, providing flexibility for changes during the design process, crucial in an industry often characterized by spontaneous ideas and creative adjustments.
Simulation of Light and Effects: Virtual modeling allows the simulation of lighting conditions and effects, aiding in understanding and optimizing the visual impact of the stage set under different conditions.
Archiving and Reuse: Digital models can be archived and reused for future productions, saving time and resources for upcoming projects.

Overall, virtual modeling for stage sets offers a contemporary and innovative approach that not only optimizes workflow but also provides ecological and economic benefits.

3D-Programs for stage design

Complex programs like Autodesk’s AutoCAD enable the design and visualization of ideas for a stage production, supporting the process from design through modeling and prototype creation to production. Both 2D and 3D renderings can be designed for better concept communication. However, as AutoCAD focuses heavily on technical aspects and offers numerous features, it may not be suitable for quickly creating stage set concepts. A program with the potential to unite all elements of a stage set (construction, materials, lighting) and make them easily adjustable would be beneficial.

Im/material Theater Spaces

A subproject of the research project „Im/material Theater Spaces“ focused on developing methods and tools for virtual construction rehearsals that can take place in the virtual space and are location-independent.

During the construction rehearsal, the artistic and technical teams come together for the first time in a large group. Here, the previously submitted stage set design model is marked on the stage to visualize ideas and dimensions in a 1:1 scale. Through discussions, the technical feasibility is examined. It is crucial that during the rehearsal, the overall impression, dimension, and atmosphere of the stage set can be experienced for the first time, providing insight into the impact the stage set has on the audience. Additionally, depending on the stage set, the materiality and the use of light and projections are tested for the first time. The approach of the construction rehearsal can also be compared to Greyboxing in gaming, where relevant parts are prototypically simulated. Wouldn’t it be practical to incorporate this aspect directly at the beginning of the design and conceptualization process and use its effect as a design and idea driver?

The result of the project includes methods and tools for conducting virtual construction rehearsals. A workshop format called „How to go Virtual“ was developed to test practical applications with theaters and venues on-site. The focus was on practical application, experimenting with existing applications and programs, and playing through individual usage scenarios.

Results of the project: Virtual Construction Rehearsals: The project developed workflows and methods that serve as guides and tutorials for the theatrical landscape. Three different workflows were developed:

Sketchfab / Zoom „Semi-virtual Preliminary Rehearsal“
_Utilizes the Sketchfab platform for model presentation.
_Enables discussions via video conferences and shared screens.
_No live editing of the 3D model during the discussion.
Mozilla Hubs „Rehearsal for Everyone“
_Uses the Mozilla Hubs platform for virtual rehearsals.
_Easy accessibility for up to 30 people.
_No live editing of the 3D model during the discussion.
Virtual Construction Rehearsal with VR-Sketch
_Utilizes the paid plugin VR-Sketch for SketchUp.
_Enables discussions and construction rehearsals via VR headsets.
_Live editing of the virtual 3D model during the discussion.

Thinking further, a virtual construction rehearsal could not only include the stage set itself but also integrate costume design, with 3D elements related to the stage set. I came across the research project „connecting.stitches“ by Luside Ehrenwerth, where she combines costume design and technologies. Her focus is on the possibility of experimenting with conductive fabrics, 3D printing, and sensors before transferring them into production. This free experimentation could be directly incorporated into the design of an entire scene in VR to examine the overall concept more precisely. This way, all products from different workshops could be coordinated to be viewed in exact scale.

Finally, it’s worth noting that real materials, paper, and pencils are often used for sketching ideas and concepts. Could it be a possible approach to take these 2D representations as they are and transport them directly into the virtual 3D space to make quick decisions for the progressing design process?

Institutions

Deutsche Theater Gesellschaft – Services

Ressources

by david.lasslberger - 4. Dezember 20234. Dezember 2023

#04 What is „immersion“?

As I was thinking about what to write the next blog post about and in which direction to take my research, my attention got caught by a simple word: immersion. We hear and use it all the time. Every company wants to create immersive experiences with their marketing, and we say „that video game is immersive“.
But what does it mean really?
Why are some things considered immersive, and why aren’t others? And is it important when you think about therapy? Does it have to be immersive to produce therapeutic results?

When you ask google for a definition about immersion, the first result explains it like this:

the action of immersing someone or something in a liquid.
„his back was still raw from immersion in the icy Atlantic sea“

deep mental involvement in something.
„a week’s immersion in the culinary heritage of Puglia“

Oxford Languages Disctionary

For our case, the second definition is the far more interesting one. When we think of immersive experiences and what are also the first search results to come up are modern digital experiences. For example, light projection mappings in museums, VR/AR/XR experiences, video games, etc.
What I found interesting is that the definition characterizes it as „deep mental involvement in something“ and that something may be anything. We can find ourselves immersed in a book, for example, where there is no other external stimulus other than the words on the page. However, the images we create in our head, our imagination, can immerse us in these stories. We are solely focused on the story told, and we do not notice how much time passes. This „flow state“ may also be the reason why we can get immersed in work as well.

In the modern media landscape, immersion is also often used to describe interactive media. Media forms where the viewer also becomes a participant and can influence the media in some way, i.e. the viewer has some level of autonomy in what is created. I mentioned some types like these above. For example, video games, light mapping installations, virtual reality applications, etc.

One concrete example would be the ARTE museum in Gangneung, South Korea. Here visitors find themselves in environments made up of huge displays and light mappings where they can get lost in otherworldly surroundings. Through both visual and auditive stimulation the visitor can feel completely involved i.e. immersed.

When it comes to therapy, the term „immersion therapy“ crops up. This describes a more intense form of exposure therapy. Depending on the devices used it may also be called „virtual reality exposure therapy“. The patient is put into a virtual environment which can be used to, in a very controlled way, expose him*her to their fears. Because the patient is more stimulated/immersed than via more traditional methods, this form of therapy seems to prove quite effective. This immersion is especially useful when treating mental problems such as PTSD or phobias, but may be less necessary when dealing with physical ailments.
This topic of „immersion therapy“ is a vast and very interesting topic and this has just been a short overview (but it might be an interesting topic for further exploration. Maybe in an upcoming blog post…).

To summarize, immersion is something that happens when we are deeply mentally involved in something. This can be when we are stimulated by our own thoughts (e.g. reading a book) or from exterior stimuli (e.g. video game: visuals, audio, touch of input device, …). For therapy, it can help to make these processes more intense but also controllable.

References used in this article and for research

What is Immersive Media: An Introduction – XR Today

https://www.miamiherald.com/entertainment/visual-arts/article282395128.html?taid=65653df5b51a5b0001a53a8c&utm_campaign=trueanthem&utm_medium=social&utm_source=twitter

ARTE Museum Gangneung

ODYSSEY VISUAL MEDIA – YouTube

Immersive Museum Experience – YouTube
The Longest Night illusionist exhibition ankara

Immersion Therapy for Treatment and Support of Mental Health conditions (simulationmagazine.com)

What Is Virtual Reality Exposure Therapy? (choosingtherapy.com)

Immersion Therapy For Anxiety: How It Works And Techniques (mantracare.org)

Immersion Therapy vs. Exposure Therapy – Healthy Minded

What Is Exposure Therapy? (apa.org)

by vinzenz.weber - 26. November 2023

XR 3 // VR Interactions: Controller vs Body Tracking

Virtual Reality (VR) has revolutionized the way we experience digital content, offering immersive and interactive environments. One important aspect of VR is the method of interaction, which can greatly impact the user experience and effectiveness of training applications. In this blog post, we will delve into the differences between controller-based and body tracking interactions in VR, their implications for training, and recommendations for interaction design.

The Importance of Natural Interaction

Research has shown that natural and intuitive interactions in VR enhance presence, immersion, and usability. When users can interact with virtual objects in a way that mimics real-world actions (e.G. by actually grabbing objects or by pushing Buttons with the virtual controllers), they can focus on the learning experience rather than learning how to use the equipment. This allows for a deeper level of engagement and better retention of information. As Abich and colleagues (2021) argue, training in VR is most useful when it allows for embodied, experiential interaction. The spectrum of interactions in VR ranges from „arbitrary“ interactions (e.g., double-clicking a mouse to select an object) to „natural“ interactions (e.g., rotating the hand clockwise to rotate a valve in the same direction). The more natural an interaction seems, the higher presence, immersion, and usability it affords.

Controller-Based Interactions

Controller-based interactions in VR function similarly to game console controllers. The difference lies in the tracking capabilities of VR controllers, which accurately represent the user’s hand position and movement in the virtual environment. This enables users to interact with virtual objects in a more precise manner. However, a challenge with controller-based interactions is the lack of standardized control schemes. Different VR applications may require users to interact with virtual objects using various methods, such as reaching out or using a laser pointer. This inconsistency can lead to confusion and cognitive load for users. It is important for developers to consider implementing standardized control schemes to provide a consistent and intuitive user experience across different VR applications.

Physical Button in virtual Space
https://www.youtube.com/watch?v=lPPa9y_czlE

Pointer Device
https://www.youtube.com/watch?v=8fT478uopco

Body Tracking Interactions

Advancements in technology have enabled body tracking interactions in VR, such as eye tracking and hand/body tracking. Hand tracking, in particular, allows users to interact with virtual objects in a more natural and intuitive manner. For example, using pinch gestures to select objects or pressing virtual buttons. Hand tracking can enhance the sense of presence and immersion in VR experiences. However, similar to controller-based interactions, there is a lack of standardized gestures for hand tracking. Users may need to learn specific gestures for different applications, which can be a barrier to intuitive interaction. To address this challenge, companies like Ultra Leap are working towards establishing standards for gesture controls in VR. By introducing standardized gestures, users can intuitively know how to perform actions without the need for extensive tutorials or guesswork.

Recommendations for Interaction Design

To optimize the user experience in VR, it is crucial to provide clear and concise tutorials for both controller-based and body tracking interactions. Instead of relying on textual instructions, show users how to perform actions through visual cues and demonstrations. This „show, don’t tell“ approach can help users quickly grasp the interaction methods without the need for extensive reading or trial and error. Additionally, consider placing important buttons and menus within reach of the user to minimize the need for physical movement within the virtual environment. For example, utilizing a popup menu on the wrist of one virtual controller can provide quick access to important functions without requiring users to navigate across the virtual room. These design considerations can help reduce cognitive load and ensure a seamless and intuitive user experience in VR.

In conclusion, the choice between controller-based and body tracking interactions in VR depends on the specific application and the desired level of immersion and naturalness. Both interaction methods have their advantages and challenges, but with thoughtful design, standardized gestures, and concise tutorials, VR can offer truly immersive and effective training experiences. As VR technology continues to evolve, it is important for developers and researchers to collaborate in establishing best practices and standards for interaction design to unlock the full potential of VR for training and other applications.

Links

by leonie.dunke - 20. November 202330. Dezember 2023

2 | Theatre in the digital time

This week, my main focus was on exploring additional stage performances incorporating new technologies in their design. Since Virtual Reality (VR) and Augmented Reality (AR) are still relatively new mediums in classical theater, I dedicated my focus to them this week.

During my research, I came across the State Theatre Augsburg, which has developed an intriguing concept as the „5th department.“ With their Digital Theatre, they are experimenting with new forms of staging, aiming to make high-tech accessible. A key element in this initiative is the use of 500 VR goggles, not only within the auditorium but also available for loan nationwide in Germany and Austria. To foster a scientific exchange on the topic of digitization in theater, they launched the initiative theaterentzwerk.digital. The theater has already produced several pieces, which I will briefly discuss. ¹

Elektrotheater:
Christian Schläffer and Daniel Stock created a VR experiment that can be experienced from the comfort of one’s sofa. Technically, it’s a multiplayer online game where the audience and artists appear as avatars, interacting live. The viewer goes through various scenarios, aiming to bring social interaction into the digital realm. Gesture communication with other viewers and direct interaction altering the space are key components. Playful instructions are given to the audience, guiding them through the space. ²

Shifting Perspective:
This involves the use of motion capturing. Two dancers perform short, specially created sequences around the viewer, playing with the distance between the dancer and the observer. Due to the 360-degree perspective, the viewer must rotate and set their own focus

Nikolai Gogol: The Employee – Diary of a Madman:
Here, holograms are used in the VR world

Orfeo ed Euridice (2020):
A combination of opera and VR using 500 VR goggles. The audience could put on the goggles during specific passages and immerse themselves in virtual reality while live music played

Another pioneer is the Royal Shakespeare Company, which uses technology as a means to an end. With their online live performance Dream (2021), they reached 65,000 interested individuals from 90 different countries. This form of theater has the potential to allow international plays to be viewed regardless of location and time. Dream was created due to theater closures during the COVID-19 pandemic. The audience could open the Dream website on their mobile phones, laptops, or tablets and directly influence the performance in real-time, changing not only the movements of the artists but also the music. ³

The use of VR technology provides the audience with an immersive experience and the opportunity to dive into the stage’s world. However, how do artists experience this form of theater? They never see their audience, and the traditional applause at the end of the performance is absent. How might artists perceive this

The use of Virtual Reality in theater allows for intimacy through proximity and breaking free from one’s own four walls. This form of theater can reach a new audience, promote the concept of home theater, and actively involve the audience in the events. With VR in theater, gameplay takes center stage: How does the audience interact with the artists? How does the audience navigate through the space?

Sonne | Luft

Furthermore, I attended the play „Sonne/Luft“ at the Schauspielhaus Graz to explore the use of AI and large-scale projections in theatrical performances. During the play, I contemplated to what extent these new media support the narrative

Brief synopsis:
„From a spaceship in outer space, society looks down upon the dilapidated planet Earth. It is not only physically distant; its existence is also temporally in the past. As people on their journey attempt to preserve what has long slipped through their fingers—an untouched environment, genuine humanity—the irreversible consequences of the climate catastrophe unfold. Nature retaliates, indifferent to humanity’s demise.“ ⁴

Technologies used:
Almost the entire stage setup was placed on a rotating stage that constantly revolved, showcasing various rooms of a space station. Ninety-five percent of the action took place on this revolving stage, with only occasional glimpses into other spaces. A large-scale projection served as the backdrop, depicting the journey through outer space. AI was utilized in the creation of film material, presenting absurd scenes that accompanied the play’s themes. The film material was integrated into the stage design through additional screens. As a final medium, an iPad was incorporated into the performance, allowing for the selection of film material

Conclusion:
Time was significantly slowed down. The actors moved in almost slow motion, mirroring the rotation of the stage and the gradually changing background projection. The substantial slowing down of events both in content and visually manipulated the audience’s perception. The actual transformation was imperceptible until it was too late, and suddenly, the audience found themselves in an entirely different situation. Without a dynamic stage design, this gradual change would not have been possible.

Institutions

Staatstheater Augsburg – Digitaltheater

Resources

by david.lasslberger - 19. November 202319. November 2023

#02 Diving into Virtual Reality

Like I said in the first blog post, I want to use these posts as a means to delve into different topics. I look forward to widening the scope of this topic. But before that, I want to note down a specific topic I have in mind. Virtual reality therapy. I will try to keep this as concise as possible. Therefore, without further ado, let us jump right into it.

What is Virtual Reality?

Before delving into the subject matter, it is essential to establish a shared understanding of the term Virtual Reality (VR). The Oxford English Language dictionary defines it as such:

The computer-generated simulation of a three-dimensional image or environment that can be interacted with in a seemingly real or physical way by a person using special electronic equipment, such as a helmet with a screen inside or gloves fitted with sensors.

This is a very technical description, but it defines VR quite clearly. A VR setups is compromised of three parts: there is a computational machine which performs the calculations needed, in other words, a computer. This can be a simple image, video or entire 3D generated worlds. These 3D worlds are often created using a game engine which allows for interactive real time environments. Two notable examples which support VR would be Unity and Unreal. Then there is an output device which displays the generated images. Traditionally this would be a screen, in this case, it is a VR headset, also referred to as a Head Mounted Display (HMD). Finally, though strictly not mandatory, the inclusion of an input system may be necessary. This spans from conventional controllers to VR hand-tracked controllers. Or, as we just recently saw with Matt Corall’s presentations about Ultraleap, there is also the possibility of tracking the hands and using them for input without any controller. Haptic feedback, the simulation of touch, is also a notable component which can drastically increase the immersion and the effectiveness of VR.

However futuristic it may seem, the roots of VR extend significantly into the past. In 1838 the concept of stereopsis, the fact that the brain overlays two images to create a 3D image with depth, was first described. In the 1950s Morton Heilig created Sensorama, a device with the goal of fully immersing the user by using a stereoscopic 3D image, sound, smell, vibrations and simulated wind.

In 1960, Morton Heilig, the innovator behind Sensorama also patented the Telesphere Mask, which can be considered the first HMD. Skipping ahead, in 1997 Georgia Tech and Emory University collaborated to utilize VR as a therapy method for the treatment of Post-Traumatic Stress Disorder (PTSD) in war veterans.

With this short history overview, I wanted to shake the notion that VR is something entirely new. While I provided a brief glimpse, I glossed over many other captivating inventions. If you are interested, I recommend having a read of the full articles – they really are fascinating. The links can be accessed in the Sources section below.

Use cases in therapy?

Though there is often an overlap I would differenciate between two different use cases:

physical
mental

Let us begin by considering the physical use case,. For example, partial paralysis of a body part or side. While conventional treatment methods exist, Virtual Reality (VR) presents distinctive advantages. The therapy experience can be tailored to exact use cases, which would be hard to train reliably in real life scenarios, such as relearning how to drive. A driving simulator setup, i.e. a chair with a steering wheel and shift lever should not be used instead of VR, but they should work in tandem to increase immersion and effectiveness. Furthermore, the experience may easily be gamified, meaning turning the process of therapy into a fun game. This may especially help when dealing with children who may not have the discipline or motivation to push through rigorous training programs.

VR is also especially useful in the treatment of mental problems. A notable use case involves the treatment of specific phobias, including but not limited to the fear of flying, arachnophobia, elevator anxiety, or social anxiety. Treating a fear like flying is difficult with more traditional treatment methods. Arranging a plane, traveling to a specific location, and repeating such processes multiple times can be logistically difficult and time-consuming. Using VR, a 3D scene can be comparably easily created and the treatment can be done in a very controlled fashion. Furthermore, as previously mentioned, VR has been employed by the military for the treatment of war veterans grappling with Post-Traumatic Stress Disorder (PTSD).

BraveMind – a VR treatment method for war veterans struggling with PTSD

In this context it is used to allow soldiers to relive the traumatic experiences and work through them with a specialist, in a carefully designed and controllable manner. Soldiers who may not be able to cope with their experiences may decide to commit suicide as a result. Therefore application of VR in trauma-focused therapy provides a crucial and potentially life-saving intervention for individuals dealing with the profound impact of their military service.

Personal experience

In my previous blog post, I delved into my personal motivations surrounding this topic. Since then, I have talked with my brother and the kind of experiences he had using VR therapy. In his particular case, VR served as a tool for training the left side of his body, which experienced partial paralysis, resulting in reduced speed. Additionally, VR was employed to address issues related to his partially impaired field of vision.

He recounted three different programs which were used in his treatment. Firstly, a car driving simulator, which was used to train both his motor function and his ability to perceive traffic. A virtual room in which he needed to search for objects and, on occasion, connect different objects using wire. And lastly, a game in which balls were being thrown at him, and he had to deflect them using his hands. He expressed a strong preference for the visual feedback of seeinghis hands in the VR environment. He also noted that he talked quite a lot with his therapist and at least in Austria, the options of VR treatment programs is very limited. Few programs exist and they can not really be customized to the needs of the user. In the last example, my brother wanted to train his left side more but have the objects be slower and the therapist said that this cannot be changed, unfortunately. This seems to be a common problem with these programs – the customization options for individual patients is limited.

Summary

In summary, VR has a long history, yet its potential as a treatment method remains underutilized, presenting a lot of potential for innovation and research in this area. The versatility of VR spans both physical and mental health topics. VR therapy proves useful because it can be individualised and is both time- and cost-effective. As we continue to

, offering the distinct advantage of tailoring experiences to individual users while proving to be time- and cost-effective. As we continue to uncover the multifaceted applications of VR in the realm of therapy, its transformative impact on healthcare interventions is poised for further realization and advancement.

Sources

The history of virtual reality
History Of Virtual Reality – Virtual Reality Society (vrs.org.uk)
History of VR – Timeline of Events and Tech Development (virtualspeech.com)

BraveMind video
Virtual Reality Therapy: PTSD Treatment for Veterans (soldierstrong.org)

by vinzenz.weber - 19. November 2023

XR 2 // The Evolution and Current Landscape of Extended Reality

From Sensorama to Apple Vision Pro: A Journey Through XR’s History

The Beginnings: Sensorama and the First HMD

The journey of Extended Reality (XR) dates back to 1956 when cinematographer Morton Heilig created Sensorama, the first Virtual Reality (VR) machine.

This innovative movie booth combined 3D, stereoscopic color video with audio, smells, and a vibrating chair, immersing viewers in a unique cinematic experience. Heilig’s pioneering work didn’t stop there; in 1960, he patented the first head-mounted display (HMD), merging stereoscopic 3D images with stereo sound, laying the groundwork for future VR technologies.

Early Steps in Augmented Reality: The Sword of Damocles

By 1965, the field of XR took another significant leap with Ivan Sutherland’s development of „The Sword of Damocles“ Considered the first augmented reality (AR) HMD and tracking system, it aimed to enhance users‘ perception of the world. Despite its primitive user interface and simple wireframe graphics, it marked a crucial step in the evolution of AR.

Mediated Reality and the Reality-Virtuality Continuum

In the 1970s, Steve Mann’s research into mediated reality, which later influenced tech giants like Google, Apple, and Microsoft, focused on augmenting human perception through digital overlays in the real world. Building on this concept, Paul Milgram and Fumio Kishino introduced the Reality-Virtuality continuum in 1994, illustrating a spectrum of experiences from purely real to purely virtual environments.

The 1990s: Pioneering AR and the Birth of Sportsvision

The 1990s saw Thomas Caudell & David Mizell develop the first see-through HMD, coining the term „augmented reality.“ In a significant mainstream breakthrough, Sportsvision broadcast the first live NFL game in 1998 with a yellow yard marker overlay, revolutionizing sports broadcasting.

Modern Advancements: The 2010s Onward

The 2010s heralded rapid advancements in XR technology. Key developments included:

First Oculus Rift Prototype: A milestone in VR technology.
2014 – A Landmark Year: Sony and Samsung jumped into the VR headset market, while Google launched the affordable Cardboard VR viewer and the Google Glass AR glasses.
Microsoft’s HoloLens: Released in 2016, it introduced a more interactive AR experience, often referred to as „mixed reality.“
Pokémon GO: This 2016 game brought AR to the masses, demonstrating the technology’s mainstream appeal.

A Push into the Mainstream

Apple’s AR Kit and Google’s MR Toolkit made AR accessible on smartphones, broadening the technology’s reach. In 2017, the IKEA Place app showcased AR’s practical use in retail, allowing users to visualize furniture in their homes before purchasing.

The Current State: Meta Quest 3 and Apple Vision Pro

Today, we see state-of-the-art AR and VR combinations through devices like Meta Quest 3. The recent announcement of Apple Vision Pro signals a potential expansion in audience reach, acceptance, and continued research and development in mixed reality technologies.

Links

by dragana.maksimovic - 19. November 202320. November 2023

Can Virtual reality help children with ASD?

In my second week of research, I have decided to focus on autism spectrum disorder (ASD) as one of the cognitive disabilities I wish to address.

When working with children who have ASD, teachers have to have a very careful approach and special strategies that reflect needs of each child individually. Children with autism often face many challenges adapting to new and unfamiliar environments in their everyday lives. This can cause a lot of anxiety and stress, which leads to loss of confidence and avoidance of social communication and interaction.

One of the topics that sparked my interest the most, when it comes to tools for helping children with autism, was the use of VR for educational purposes. VR can have many benefits in supporting better learning and educational development of children with autism. Many studies have shown that VR can be a useful educational tool for all children, improving education and giving better results than some traditional teaching methods used in education.

Some of the most important benefits for children with autism are: