SaccadeNET: Gaze prediction for Infinite walking in VR | Patent

Yashas Joshi

0

3D Designer

AR/VR Developer

Game Developer

Blender

C#

Unity

Mother Dragon
Mother Dragon
Navigating virtual environments (VE) becomes a challenge when they are spatially larger than the available physical tracked space (PTS). Redirected walking (RW) wraps the VE and redirects the users away from the PTS boundaries as a solution to this dilemma. However, the main challenge is ensuring that the interference with the VE is imperceptible to the user.
Watch on YouTube
This paper proposes a novel RW technique that leverages change blindness induced due to saccades during a head rotation. However, unlike state-of-the-art, our approach does not impose additional hardware requirements for eye-trackers. Instead, SaccadeNet, a deep neural network, is trained on head rotation data and predicts saccades in real-time for the head rotations exceeding the velocity of 150o/sec. Rigid transformations are then applied to the VE for redirection during the onset duration of these saccades.
A straight virtual distance of 42.3m traveled by a user from within a 3.5x3.5 m2 of physical space.
A straight virtual distance of 42.3m traveled by a user from within a 3.5x3.5 m2 of physical space.
We present three user studies. The relationship between head and gaze directions is confirmed in the first user study, followed by the training data collection in our second user study. Then, after some fine-tuning experiments, the performance of our redirection technique is evaluated in a third user study. Finally, we present the results demonstrating the efficacy of the proposed technique that successfully redirects users, allowing them to walk up a straight virtual distance of at least 38 meters from within a PTS of 3.5×3.5m2.

Publication

Additional Images

The Test Environment
The Test Environment
Ariel Enemy
Ariel Enemy
Ground Enemy
Ground Enemy
Collectible Crystal
Collectible Crystal
Like this project
0

Posted Mar 30, 2024

Developed a deep-learning powered redirected walking system that allows users to walk long straight distances in VR from within a 3.5x3.5m2 of physical space.

Likes

0

Views

4

Tags

3D Designer

AR/VR Developer

Game Developer

Blender

C#

Unity

Yashas Joshi

AR/VR Developer | Ph.D. | Fast & Reliable 🌐

ChatGPT-powered AR Personal Assistant
ChatGPT-powered AR Personal Assistant
Real-time Flood Sim for Risk Assessment and Prevention | UE5
Real-time Flood Sim for Risk Assessment and Prevention | UE5
Dynamic Foveated Rendering for Redirected Walking in VR | Patent
Dynamic Foveated Rendering for Redirected Walking in VR | Patent
Project iMareCulture | Funded by EU Horizon 2020
Project iMareCulture | Funded by EU Horizon 2020