SaccadeNET: Gaze prediction for Infinite walking in VR | Patent

Navigating virtual environments (VE) becomes a challenge when they are spatially larger than the available physical tracked space (PTS). Redirected walking (RW) wraps the VE and redirects the users away from the PTS boundaries as a solution to this dilemma. However, the main challenge is ensuring that the interference with the VE is imperceptible to the user.

This paper proposes a novel RW technique that leverages change blindness induced due to saccades during a head rotation. However, unlike state-of-the-art, our approach does not impose additional hardware requirements for eye-trackers. Instead, SaccadeNet, a deep neural network, is trained on head rotation data and predicts saccades in real-time for the head rotations exceeding the velocity of 150o/sec. Rigid transformations are then applied to the VE for redirection during the onset duration of these saccades.

We present three user studies. The relationship between head and gaze directions is confirmed in the first user study, followed by the training data collection in our second user study. Then, after some fine-tuning experiments, the performance of our redirection technique is evaluated in a third user study. Finally, we present the results demonstrating the efficacy of the proposed technique that successfully redirects users, allowing them to walk up a straight virtual distance of at least 38 meters from within a PTS of 3.5×3.5m2.