Searching for Sound Engineers

Music-Driven Web Experience — Emotion-Based UX System

One-liner: A web experience where the currently playing track controls the visual environment, creating a dynamic interface that responds to sound as a first-class design input.

Problem

Music platforms treat audio and UI as separate layers — you hear a song while looking at static metadata and album art. There is no system that allows sound itself to become a generative design material, leaving a significant emotional bandwidth on the table.

Solution

The system treats audio as a live data source. As music plays, extracted properties — tempo, frequency range, energy level, mood classification — drive UI state in real time. Color, motion, layout density, and interaction behavior all respond to what is playing. The experience is not visualized audio; it is an interface that feels like the music.

Experience System

Audio → State: The audio engine extracts properties per track (BPM, energy, key, mood). These map to a set of defined UI states — not a 1:1 linear translation, but a curated system of thematic environments.

State → Visual: Each UI state has a corresponding visual language: color temperature, typography behavior, background motion, and element density. A high-energy track produces a compressed, kinetic interface. A slow, low-frequency track expands the layout and reduces motion.

Visual → Interaction: User interactions (hover, scroll, click) are modulated by the current audio state. Hover effects are faster in high-BPM states, slower and more diffuse in ambient states. The interface is never static — behavior is continuous and reactive.

Key Interaction Moments

Track Transition — When a track changes, the UI transitions through a bridging state rather than cutting abruptly. Color and motion shift over 1–2 seconds, matching the audio's fade behavior.

Energy Peak — At detected energy peaks (chorus, drop, climax), a brief full-screen pulse or layout shift signals the moment without interrupting the experience.

User Hover / Explore — Hovering over track metadata or navigation elements reveals information through motion rather than a static tooltip. Reveal behavior scales with current audio energy.

Silence / Pause — When audio stops, the UI enters a resting state — reduced contrast, minimal motion, slow breathing animation. The interface communicates absence without becoming dead.

Visual / Aesthetic Direction

The visual language is defined by restraint with controlled moments of intensity. The base palette is near-neutral — dark ground with desaturated tones — allowing color to carry full meaning when audio state triggers it. Typography shifts between a geometric grotesque (high-energy states) and a high-contrast serif (ambient states), reinforcing mood through form. Motion is physics-based: easing curves match the emotional texture of each audio state rather than defaulting to ease-in-out uniformity.

Technical Considerations

Audio Analysis: Web Audio API for real-time frequency and amplitude data; track metadata (BPM, energy, key) sourced from a music intelligence API (Spotify Audio Features or equivalent) for non-real-time properties.

State Management: A finite set of UI states (~5–7) maps to audio property ranges. State transitions are debounced to prevent thrashing on rapid audio changes.

Performance: CSS custom properties (--energy, --tempo, --mood-hue) are updated by a single JavaScript loop, keeping layout-triggering reflows out of the animation path. All background motion runs on the compositor via transform and opacity only.

Responsiveness: The system degrades gracefully on mobile — motion is reduced, layout simplifications are applied, and battery-sensitive devices receive a low-motion mode via prefers-reduced-motion.

Accessibility: Audio-reactive motion respects the OS-level reduced motion preference. Color contrast is validated against WCAG 2.1 AA at all UI states, including high-saturation peak moments.

Outcome / Impact

The experience creates a demonstrably higher sense of immersion and presence compared to static music interfaces — users spend more time in active listening states and engage more with track discovery. The system design is reusable: the audio-to-UI-state mapping layer can be extended to any visual theme without rearchitecting the core engine. As a portfolio piece, it demonstrates mastery of interaction design, real-time system behavior, and the use of sensory input as a first-class UX variable.