The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
šµ Play Musical Notes with Hand Gestures (Sound ON!)
Check out the project:
š Builder.io Entry (Mouse & Touch experience ā don't forget to click the pads!)
š Live Demo on Vercel (Full Camera + Audio experience)
Hi! š Iām Kirill from Berlin. Video processing seemed hard, so I asked AI to help, and built this in just two prompts! It generated the functional prototype, letting me focus on the design.
How: 100% local (MediaPipe + Web Audio). Zero latency & privacy.
Note: Builder.io iframes block cameras. Use the Vercel link for the full experience!
For me, as someone who has no musical education, it was surprisingly to get caught up in the process. I even made something catchy (no proofs will be provided haha).
Tracking accuracy is pretty spot on!
This makes me think of the movie, Minority Report from 'back in the day' and you made the technology come to life in 2025. Well-done! :)
Thanks a ton for the awesome comment <3 !
Old movie aesthetic š„ ā interfaces were just blank canvases back then, before all the standard corporate computer interactions.
Definitely rewatching it this weekend! :)
Yeah, I remember how antiquated computers were back then. Lol However, I think I was more specifically thinking of when Tom Cruise in the movie does some interactive hand gestures in the air to interact with some sort of government computer system. Here is clip from what I was...
Oh & you are very welcome!! .. Super cool project! :)
Pretty good!
this is the coolest thing ever, i love lightning mode, so much fun
Thanks! It's always really nice to do something just for fun, even if it's not very useful. Although I'm already thinking it would be cool to implement such a navigation feature on one of the projects.
Thank you š¤© !
ā¢ Nov 23
This is a brilliant technical feat! Using MediaPipe and Web Audio for zero-latency gesture control over musical notes is brilliant.
What was the single most challenging MediaPipe gesture stability issue you encountered that required the most time to debug or fine-tune?
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Trending
maxearnings
The next frontier of payments is live on Contra. How are you maximizing revenue?
freelancerlife
Freelancer life is wins, pivots, and everything in between. Whatās yours right now?
aidesignflow
AI tools are redefining how designer work. What does your workflow look like?
micrographics
Micrographics started as utility - barcodes, packaging, instruction labels. How would you use them?
aivideo
AI video tools are moving at warp speed. What tools are you using?