When looking into different methods of 2D and 3D character animation – be it with the goal to create a short scene or to create a responsive, engaging character – and possible workflows of character creation – the following video came up:
Video showcasing an AI-assisted keyframe-based animation workflow using Cascadeur
Thus, the idea of using AI to assist in different steps of an animation workflow was born. Hereby, the goal was to look for tools that do not completely automate the animation process, but rather that support the artist in a variety of subtle ways by automating some of the most time consuming or tedious processes of an animation workflow. The found results and software solutions will be shown in this blogpost.
AI and MoCap
It rapidly became apparent, that one of the most common applications of AI currently is in the automation of Motion Capture. Be it by using videos as a basis, employing MoCap suits for tracking or even live-tracking a movement via camera, a wide variety of softwares already use AI to automate the motion capture process and provide fully animated and rigged models at the same time or transfer the captured movement to a pre-existing model.
Radical – Real-time 3D human virtualization for everyone, everywhere.
Now, while these softwares certainly differ in a small variety of ways, such as Radical promoting their seamless integration into a wide variety of existing 3D software, Deepmotion advertising the possibility of refinement via a rotoscope editor and their big community and Plask even offering additional outsorcing service for certain tasks, their overall purpose largely remains the same – namely, to capture a real life motion and automate its transfer into a digital environment, thus cutting this part of the workflow short. Given the wide variety of tools available, the question remains, however, on how to proceed from there or how to create new animations from scratch without using motion capture?
AI assisting in animation
Cascadeur – AI assisted keyframe animation software
Cascadeur aims to ease the process of keyframe-animating already rigged characters by using AI to simulate existing physical forces that may influence a characters movement, such as gravity and momentum. It does so by automatically calculating the appropriate movement of all the different nodes within the rigged model in response to the movement of a single other node. The following tutorial video more accurately demonstrates, how this works:
Next steps
More research into different character-animation-workflows
Try out different workflows to see the advantages and disadvantages
Model a character, rig it and creat a small clip using the learned tools
1. Optimisation of production workflows in Western and Japanese animation: Rotoscoping, Motion Capture and other Hybrid 2D/3D Techniques
This topic deals with the question of how Western and Japanese animators and animation studios try to optimise their production workflows and how they incorporate technologies such as Rotoscoping, Motion Capture as well as other Hybrid 2D and 3D techniques into their pipeline. The aim is to provide accurate insight into different 2D and 3D production workflows and compare them along with the results they achieve. The main interest will therefore be the practical applications of such techniques and the comparison of the different results they achieve within their cultural, narrative and genre-specific context. To summarise, after an in-depth study of the material, which will hopefully show both the advantages and the disadvantages of the different approaches, the next step will be to look for ways to incorporate the learned into my own design. This will be done via different case studies and short projects, such as short animations, character models or even a short movie.
1. Agarwala, A. / Hertzmann, A. / Salesin, D. / Seitz, S.: Keyframe based tracking for rotoscoping and animation . In: “ACM transactions on graphics, 2004, Vol.23 (3)”, ACM, New York 2004 , DOI: https://www.doi.org/10.1145/1015706.101576 2. Bratt, B: Rotoscoping, Routledge, 2021, ISBN: 9780240817040. 3. Davison, A. / Deutscher, J. / Reid, I.D.: Markerless Motion Capture of Complex Full-Body Movement for Character Animation, in: “Computer animation and simulation 2001”, Springer, Vienna 2001, ISBN: 3211837116. 4. Dunlop, R.: Production pipeline fundamentals for film and game. Focal Press, Burlington 2014, ISBN: 1-317-93622-1. 5. Kitagawa, M. / Windsor, B.: MoCap for artists: workflow and techniques for motion capture. Elsevier / Focal Press, Amsterdam / Boston 2008, ISBN: 9786611307301. 6. Menach, A.: Understanding motion capture for computer animation, Morgan Kaufmann, Amsterdam 2011, ISBN: 9780123814968. 7. Lanier, L.: Professional digital compositing: essential tools and techniques. Wiley Pub, Indianapolis 2010, ISBN: 9786612471834. 8. Lin, Z.: Research on film animation design based on inertial motion capture algorithm, in: “Soft computing (Berlin, Germany), 2021, Vol.25 (18)”, Springer, Berlin 2021, DOI: https://www.doi.org/10.1007/s00500-021-06001-y. 9. Miyakoshi, M.: Correcting whole-body motion capture data using rigid body transformation, in: “The European journal of neuroscience, 2021, Vol.54”, Wiley Subscription Services, France 2021, DOI: https://www.doi.org/10.1111/ejn.15531. 10. Roberts, S.: Character Animation Fundamentals: Developing Skills for 2D and 3D Character Animation. Routledge, London 2017, ISBN: 9786613267436. 11. So, Clifford, K.F. / Baciu, G.: Entropy-based motion extraction for motion capture animation, in: “Computer animation and virtual worlds, 2005, Vol.16 (3-4)”, John Wiley & Sons Ltd., Chichester 2005, DOI: https://www.doi.org/10.1002/cav.107´.
2. Gamification and Exploration in a VR environment
The goal of this topic would be to explore the different possibilities to work, play and design within a VR environment. With a very experimental approach, the final outcome of this topic can not yet be defined, but would very likely be some form of VR environment that allows for a playful and creative exploration of different applications and possibilities that such a digital space provides. Since VR can be seen as a completely different entity from our real world allowing for the addition of a second reality right within our lived-in one, with different rules, options and limitations, the current possibilities seem almost endless and are only restricted by our imagination as designers. As such and to set a boundary for the planned thesis, the exploration of two very particular topics is of special interest to me. These are:
2.1 An integrated approach to an interactive digital VR museum Based on the current research project I am working on at the Graz University of Technology, funded by the FWF, the possibility of building a completely interactive, decentralised virtual museum within Virtual Reality could be researched. Possible topics to look into include but are not limited to: user interaction and how to increase engagements, tools to allow exploration, interfaces and information design, accessibility, visual narration and perception of space and many, many more. As such, more in-depth research of this topic could prove to be worthwhile.
2.2 Gaming and designing in / for VR Another topic that seems quite promising is inquiring how gaming and exploration can even work in a virtual reality environment, both from a design standpoint and from the view of a user. To explore the different possible playful experiences that can be created and how to create them would be the main focus of this research. By looking into existing examples, such as VR games and the user-driven and collaboratively designed environment of VR chat, a deeper understanding of the topic could be achieved, which then in turn would be used to inform the design decisions shaping the new environment created as part of this research.
All in all, this topic is sure to be the most ambiguous and open among the presented, but also the one with the most potential for playful exploration and development. As such, VR and the inquiry of possible applications, is currently my favourite.
1. Jerald, J.: The VR book: human-centered design for virtual reality, Association for Computing Machinery (pub.), Morgan & Claypool, San Rafael 2016, ISBN: 9781970001129. 2. Sherman, W.R. / Craig, A.B.: Understanding Virtual Reality: Interface, Application, and Design, Elsevier Science, Burlington 2002, ISBN: 9780080520094. 3. Caarrozzino, M. / Bergamasco, M.: Beyond virtual museums: Experiencing immersive virtual reality in real museums, in: “Journal of cultural heritage, 2010, Vol.11 (4)”, Elsevier SAS, 2010, DOI: https://www.doi.org/10.1016/j.culher.2010.04.001. 4. Lengua, D. / Lengua, M.: Moving Towards Digital Lands: on Processes of Interaction in Museum’s Virtual Spaces, in: “Meta space”, DISTANZ Verlag, Berlin 2022. 5. Plowman, J.: Unreal engine virtual reality quick start guide: design and develop immersive virtual reality experiences with unreal engine 4, Packt Publishing Ltd, Birmingham / Mumbai 2019, ISBN: 1-78961-504-6. 6. Mitsea, E. / Drigas, A. / Skianis, C.: VR Gaming for Meta-Skills Training in Special Education: The Role of Metacognition, Motivations, and Emotional Intelligence, in: “Education sciences, 2023, Vol.13 (7)”, MDPI AG, Basel 2023, DOI: https://www.doi.org/10.3390/educsci13070639. 7. Evans, L. / Rzeszewski, M.: Hermeneutic Relations in VR: Immersion, Embodiment, Presence and HCI in VR Gaming, in: “HCI in Games”, Springer International Publishing, Cham 2020, DOI: https://www.doi.org/10.1007/978-3-030-50164-8_2. 8. Yildirim, C.: Cybersickness during VR gaming undermines game enjoyment: A mediation model, in: “Displays, 2019, Vol.59”, Elsevier B.V, 2019, DOI: https://www.doi.org/10.1016/j.displa.2019.07.002.
3. 3D character modelling and narration in Virtual YouTubing / the V-Tuber-community: How to make your character engaging via VFX and animations to underline your narration
Virtual YouTubing (V-tubing) has evolved over the recent years from a small niche community into an active and dynamic format of online content creation, where content creators interact with their audiences through animated 3D avatars. This topic delves into V-Tubing as a form of content creation, focusing on its combination of 3D character modelling and rigging and the narration and storytelling behind the characters of different creators. The main inquiry will be, how content creators use visual effects (VFX) and animations to create engaging characters that line up with their created narratives.
V-tubers, in this context, balance different skillsets, between technology and its capabilities to entertain and the storytelling aspect of their character. It is not surprising then, that some creators instead already opt to outsource one or the other, sometimes even both, to a digital artist, since the skillset required is expansive and the quality of many models in the competition is on the rise. Nonetheless, there is also still those that do everything themselves. The research aims explores the intricacies of character creation, using 2D character designs in combination with 3D modelling and rigging to craft distinct personas and then employing advanced animations and VFX to enhance their narrative expressiveness. By doing so, a deeper understanding of the complexity these creators are faced with, can be achieved.
The proposed research will then go on to explore the possibilities of viewers developing emotional attachment towards these digital personas, examining how the interplay of animations and VFX may enrich the viewer experience and engage the viewer. It will include studies into collaborations and manners of fan engagement – for example, fan art – that acts as the foundation of this community.
Finally, this topic aims to create one such V-Tuber model from scratch, going through all the most common steps from first draft to finished character. By doing so and by documenting the process, it will hopefully act as an investigation into the role of technology in facilitating the creation of these characters, including the software and hardware used to bring V-Tubers to life.
