SWAPBLOCK — an AI hand-tracked voxel world
I built SWAPBLOCK for the Flowstep Challenge: a browser-based voxel game you control with nothing but your hand and a webcam. No mouse, no keyboard.
You pinch your thumb and index finger together to grab a colored block, move your hand through the air to carry it, and release to drop it.
The goal: match each block to the colored ring that corresponds to it, using only hand gestures tracked live through your camera.
Design process: I designed the entire multi-screen product, landing, onboarding, live session, and results screen in Flowstep, working through the color system, typography, and interaction states screen by screen before writing a line of code.
Flowstep let me iterate on the whole product experience fast: refining the HUD layout, the gesture-status states, and the color-matching mechanic visually before I touched implementation.
From there, I built the functioning version using Three.js for the 3D voxel world and MediaPipe Hands for real-time in-browser hand tracking.
No AI training required; everything runs client-side. Sound is fully synthesized in-browser (no audio assets), and the whole thing is deployed live on Vercel.
>>> Flowstep file (design prototype): https://app.flowstep.ai/file?activeFileId=e6f9d351-0366-4b64-a3d5-63f31bbf1a2b
>>> Live, playable demo: https://swapblock.vercel.app/
Happy to walk through the process
2
85
Aetheria website Design (Framer Development)
1
3
Lumee website (Framer Development)
0
12