New Technology Enables Real Walking Around in Virtual Reality

The door is open, a fireplace crackles on the right, the sun is shining outside and butterflies flutter by. The floor is covered in clay slabs, the ceiling is made of wood and hefty beams support the roof. Only a few steps and you are standing on the patio, gazing at the trees and green hills. After a brief pause, you walk around the corner of the country house, where you are greeted by a spectacular view from the cliffs down to the sea, with gulls circling above the waves. Then you take off your glasses. Instead of a country estate in Italy, you are standing in a seven-by-twelve-metre room in the cellar of the CLA Building at ETH Zurich with black floor slabs, bare grey brickwork and small strip-lights beaming down coldly from the ceiling. The illusion of having a quick stroll around a house in Tuscany was almost perfect.

Real walking in a virtual world

Thomas Nescher, a doctoral student at the Innovation Centre Virtual Reality (ICVR) supervised by Andreas Kunz at the Institute of Machine Tools and Manufacturing (IWF), has spent the last few years developing a novel system that makes real walking possible in virtual environments (ReWaVE), enabling the user to explore a virtual environment without any perceived restriction – by actually walking around a considerably smaller real room.

Virtual environments are nothing new in themselves. Flight simulators, computer games such as Sims or even rehabilitation systems have enabled people to plunge into digital realities to learn particular skills, for instance. And there are already systems that simulate real walking through the use of treadmills. Such systems, however, are expensive and do not convey the sense of actually walking.

Almost perfect illusion thanks to clever algorithm

In order to go for a walk in the virtual world of ReWaVe, comparatively little equipment is required: a head-mounted display (HMD) with a position measurement device pointed at the ceiling, a laptop carried on a frame, three round, black-and-white reference marks per square metre and the algorithm developed by Nescher.

The ceiling markings help the measuring device on the HMD to detect the position of the user and his or her head movements. This data is then analysed on the laptop in real time. Without any noticeable delay, the walker sees the virtual environment that the computer has calculated and loaded in the HMD.

Cunning redirection

As long as the virtual room is not larger than the actual one, it can be explored without any trouble. “If you want to be able to walk around a larger virtual world in a limited space, however, you need a few tricks,” says Nescher.

The most important of these is so-called redirection, which prevents the user from bumping into the wall in the real world. To enable him or her to walk around the virtual environment as untroubled as possible, different redirection methods have to be planned intelligently in advance. For this to work, the system needs to know where the person is in the real room and which directions of movement the virtual world offers. In principle, the “compression” of a pathway from a large virtual room to a small, real room is a control engineering problem. The algorithm anticipates the user’s future path and calculates the best way to keep him or her inside the actual room in real time.

The software uses the image information in the HMD to control the user’s movements depending on where he or she is in the real room or presumably heading in the virtual world. The path is constantly modified to prevent the user from walking into actual walls.

When turning a corner, for instance, the algorithm can make the user only move seventy or even 135 degrees instead of ninety, even though he or she still feels it as a right angle. All this results in the user following a curve in the real room that differs from the virtual route covered.

Imperceptible redirection methods are not without their limitations, however. If exceeded, the user would notice. But if unnoticeable redirection methods are no longer sufficient to keep the user in the real room, another is used, prompting the user to turn on his or her own axis. To do so, the algorithm displays an arrow pointing in the direction the user has to turn once. This enables the rotation to be increased and the user aligned optimally in the real room, which means that he or she can continue on the same virtual path without any palpable interruption.

Walk around the designs

While walking around a virtual world might merely sound like a bit of fun, Thomas Nescher can already envisage applications for his work. ReWaVe could be of interest to architects, for instance, as it would enable them to walk around their designs virtually before they are actually realised and thus iron out any weak points in their plans. And Nescher’s software also offers industrial companies an attractive opportunity to test planned production lines virtually before they build them. Until now, these companies have been producing models of assembly lines in warehouses to study ergonomic, safety-related or time-dependent aspects. Virtual environments could replace these models in future. The first firms, including Daimler and Bosch, have already shown an interest Nescher’s system.

However, the development of ReWaVe is not yet complete. The user’s legs and arms will shortly be integrated into the virtual world so that he or she can see them while walking. Nescher’s successor at the ICVR/IWF, doctoral student Markus Zank, will be taking over the reins. During his Master’s project, Zank worked on integrating realistic footstep sounds, such as the crunching of gravel, in the virtual world to heighten the sense of real walking. In future, further basic research will be necessary to study how human walking behaviour can be predicted more effectively, which should improve the redirection planning.

Reference:

 Nescher T, Huang Y-Y & Kunz A. Planning Redirection Techniques for Optimal Free Walking Experience Using Model Predictive Control. Symposium on 3D User Interfaces, IEEE, March 2014, accepted for publication.