In this blog post, I want to explain my process behind starting the prototyping phase.
A lot of considerations and decisions had to be taken into account for this project. Since it wasn’t clear to me what I wanted to do exactly, I started with a technical feasibility-first approach. It was clear that regardless of what I decided to do, I needed a way to track data and then utilize it somehow. Therefore, I decided to start by getting data from a tracking device to a game engine. I chose to use OSC (Open Sound Control) to transfer information from one device to another since we have been using it extensively during our studies, and I already have a good grasp of the concept.
For the tracking device, I chose my phone (Samsung Galaxy S23, running One UI 6.1, Android 14) and the app Sensors2OSC (not available on the Play Store, but the .apk can be downloaded from their website https://sensors2.org/osc/, and it can also be installed via F-Droid). For the game engine, I chose Unity because I am already familiar with it, whereas something like Unreal would require time to get used to.
To get the OSC data into Unity, I used an external framework. After some research, I narrowed it down to these two choices:
- keijiro/OscJack: Lightweight C# implementation of OSC server/client
- Iam1337/extOSC: extOSC is a tool dedicated to simplifying the creation of applications in Unity with OSC protocol usage
Ultimately, I decided on OscJack for several reasons. It was very lightweight (unlike extOSC, which comes with a lot of features), well-documented, and relatively new. Furthermore, keijiro has a good reputation in the GitHub/Unity community.
With this settled, I started to create a simple technical proof of concept in Unity.
OscJack comes with an OSC Connection component where you can define the type of message, host address, and port. The host is left empty because it is acting as a receive-only connection right now.
The scene in Unity is very simple. The only necessary object is the GameObject titled “OSC Event Handler,” which has the OSC Event Receiver script attached to it.
This script allows you to receive data and invoke a Unity Event that utilizes this data. For testing purposes, I created a simple script that outputs a text message and the value to the console. In this case, I used a simple int, but other data types such as Vector3 are also supported.
To quickly test if the setup was working, I created a Max8 file that sends an integer to Unity. It worked almost instantly without much troubleshooting.
It should also be noted that a dedicated OSC monitor exists, which may prove very helpful for debugging in the future. Also if anybody is wondering why my Unity windows appear reddish – it is because I set it up so the colors is changed while in play mode. Can definitely recommend so that no changes are lost.
In the final step, I added my phone to the process. I sent the data from my phone into Max8 and from there into Unity.
I probably could have done this without Max8 as an intermediary, but it is a very useful tool as it allows for rapid prototyping and easy cleanup of the data before handing it to Unity.
To have a backup system in place and to also allow me to work on multiple devices (as I work on both a laptop and a PC), I set up a GitHub repository for this project.
This was an important first step in getting a functional prototype working. In my next blog posts, I want to explore the concept and idea behind a potential prototype rather than focusing on technical details.
