The Lighthouse
When: Nov-Dec 2018 (3 weeks)
Where: Entertainment Technology Center, CMU
Roles: Game Design, Lead Programmer
Team Size: 5
Platform: HTC Vive + 5.1 Surround
Engine: Unity
The Process
We began with the question "What is the most original thing we can show guests?" We decicded to go off of a decently new, at the time, research paper on something never done before at the ETC and rarely elsewhere: climbing stairs in Virtual Reality. Planning to do this in three-ish weeks was super brash, but we pulled it off. Our efforts were rewarded with the "First Penguin Award," and we were honored to be one of sixteen teams to present at the Entertainment Technology Center's 2018 Festival.

Our test prop to simulate ascending a staircase.
The idea behind the paper was that raising and lowering your heel were the primary points of sensation for ascending and descending, respectively. For example, in the above prop, guests who walk counter-clockwise are stepping on the higher edge of the step (Rightmost in Detail B) first. Their toes go down to complete their step onto the downward angle, and their heel raises, giving the sense of ascension. We quickly solidified the prop's design based on the paper's results. While the paper was focused on linear steps, they did tease some spiral staircase designs. We went with this since it seemed more interesting for a longer experience. We modified it to fit into a sturdier, modular, and more easily reproduceable format, and set about building. The modelers soon made a 1:1 object for the game as well.

Early traversal algorithm without complete feet tracking.
To make this work, I iterated over a ton of possible locomotion solutions. The overall challenge was to make camera movement smooth while not having feet clip; the research paper failed to do the latter and did not provide info on the former. After climbing the stairs in the ETC for a while, I narrowed real movement down to being dependent on the lowest foot. You don't raise your head while first stepping up a step, but slowly while raising your back foot up to match your leading foot or the step in front of it. The first solution to come up was to not have the feet affect head movement. The head would be mapped to a spiral with linearly increasing height that followed the staircase. The head's closest point on the line would determine its height. This is the solution that ended up being the base for the final, and in the video above, althought I implemented it with angle change from a center point instead of manually calculating a good spiral function. Angle change was easier to accomodate for guests of different height.

Having this be the first didn't rule out trying others to see what felt best. One of the most notable failures was having each step be a trigger for the feet. I would keep track of which foot was "lower," and if that foot left a trigger, the head's height would increase and decrease with the foot's height until the foot was set down again. Then, the previously lower foot would either be equal height or above to the other foot, and the process would repeat on whichever foot next became lower. This was our initial solution to our clipping problem as well: we would freeze feet tracking on the non-"lower" foot on each step to prevent it from going through. This was abandoned due to difficulty in knowing the exact moment when a foot becomes "lower" in a gait.

Final feet tracking, collision, and ascension.
In the end, the virtual, tracked feet act primarily as guarantees that a player won't fall down the stairs, a pretty reasonable fear when stepping on this wood made by grad students in a couple days. At least the exposed staples and sharp edges were taken of by the final showing. Clipping was solved by separating foot tracker information from the virtual feet models. The tracked, invisible object would go anywhere in the scene, and the virtual models would follow best they can in a hacked-together inverse kinematic method. But, if the tracker was behind a wall, the model would not go through thanks to physics calculations. This gave the perfect effect of stepping making sense. The head handles all height tracking. This was smooth, but did have the unfortunate effect of making leaning forward and backward affect height. This was interesting to watch in playtests. Different guests lead with different parts of their body, some with their hips, some with their chest, others, the forehead, and so on. Smoothing out the algorithm to work with everyone was the longest part I did on the project.

As mentioned, the tracked feet were confidence ensurers. I noticed that during early playtesting, clipping feet and unreliable tracking caused people to stare down at their feet out of fear the entire experience. Making people look up and engage with the world was the largest non-technical design challenge of the project. Once feet tracking was squared away, we were free to focus on where this staircase really was.

Theming the final experience had us all in yellow raincoats.
Setting guests loose into a world of our design was fantastic.
We chose a lighthouse since a spiral staircase made sense there. Setting it in a massive storm was a great way to take advantage of our 5.1 audio, too. Given time constraints, we started our focus on interesting things that could happen while looking straight down at your feet. Rats scurry past you, a wall blows open and throws debris down past you, and so on. The crown jewel is having a step break out in front of you. Many playtesters were extremely frightened by the rickety staircase already. Having to step over a pit that you can see the depth of in VR was stressful, just the way we wanted it.

The design of the rest of the experience focused on tactility. Walking up stairs was visceral and new, so I wanted to never break that. Guests would generally take one full circle of walking to learn that their feet could be trusts, and would then be ready to move on. Then, we could hone pacing. There wasn't any method to setting the pacing of the step break, rats, and all, just grinding out playtesting and feedback. That also let me tune the feet and height changing to be much better. In the end, combining this tactile play with a fantastic soundscape created some of the best immersion I've ever done.

The Result
The final product was my favorite world I had a part in throughout BVW. At the top of the lighthouse, guests can view the wide open ocean to contrast their confined climb up and look back down at how far they've come. The metaphor for finishing the class on this project is clear. But, luckily enough, this project ended up not being done. As mentioned, we were selected to be one of sixteen out of eighty total worlds to be presented at the Festival. More information coming soon on the Festival product and the Penguin Award, seen below.

Professors, Far Left: Dave Culyba, Far Right, Jesse Schell
Team, Left to Right: Akash Phadtare, Derrick Pemberton, Julian Ochoa, Me, Liam Philiben