Gesture-Controlled Immersive Portal (Gaussian Splats)

This immersive portal system was developed as a proof of concept for a larger real-time interactive installation at Outernet London, built for a luxury real estate event. The wider project explored how web-based applications, interactive Gaussian splats, and gesture-controlled animation could come together across Outernet's screen infrastructure to create something that felt spatially novel and visually striking. For privacy reasons I'm only showing the prototype — but it reflects the core of the technical innovation behind the full installation. Built with React, Vite, TypeScript, and PlayCanvas, the system tested interaction models, scene behaviour, and input pipelines across a set of immersive portal environments. Each portal loaded a Gaussian splat scene in a specific screen, with its own shaders, textures, audio, camera settings, and interactive behaviours — all orchestrated through a shared runtime. The central concept was using gesture input to drive animation and visual change inside Gaussian splat scenes in a browser-based application. That combination was a first-of-its-kind use of web-based interactive Gaussian splats controlled through gesture-driven animation at Outernet.

A key part of the installation pipeline was connecting the system to Outernet's camera infrastructure via OSC and WebSockets, allowing live input from the venue cameras to control the experience in real time. We tested and validated the full control pipeline on-site at Outernet. The prototype also supported MediaPipe-based hand and body tracking on laptop, giving us a portable testing environment throughout development that mapped directly into the final installation workflow.

A proof of concept that formed the technical foundation of a large-scale installation at one of London's most prominent venues — and established a new format for the space: web-based interactive Gaussian splat experiences driven by gesture-controlled animation in real time.

Web-based immersive portal system built with React, TypeScript, Vite, and PlayCanvas. Interactive Gaussian splat environments with gesture-controlled shader animation, custom textures, scene-specific audio, and camera behaviour per portal. Real-time venue integration via OSC and WebSockets, connecting to Outernet's camera infrastructure for live input-driven control. MediaPipe-based hand and body tracking for portable prototype and demo workflow. Executable builds launched remotely on Outernet CPUs as part of the wider installation pipeline.

Creative Technology Development: Built the proof of concept as part of a larger interactive installation pipeline, covering real-time behaviour, interaction design, and technical feasibility. Real-Time 3D Experience: Created and integrated immersive portal scenes in PlayCanvas — scene logic, transitions, camera behaviour, shaders, and runtime control systems. Gesture-Controlled Interaction: Developed the gesture input system driving animation and visual change within Gaussian splat environments inside a browser-based application — the core of the project's technical novelty. Venue Integration: Connected the experience to Outernet's camera systems via OSC and WebSockets, enabling real-time scene control from live input within the space. On-Site Testing: Tested and validated the camera-driven interaction pipeline directly at Outernet, confirming the system held up in the venue environment. Remote Deployment: Remotely accessed Outernet CPUs to launch executable builds driving the screen experiences, layered with other visual elements across the installation. Prototype Workflow: Used MediaPipe hand and body tracking on laptop as a portable testing and demo tool throughout development, feeding into the final installation pipeline.