Vocal Parkour
A browser-based vocal-gesture rhythm game: make mouth sounds (hiss, airy, pop, hum — or record your own) into your microphone, and a fully client-side, real-time audio engine classifies each sound while a rhythm rail scores timing, precision, and consistency. Privacy-first: no audio ever leaves the device.
Key Features
Real-Time Client-Side Audio ML
A DSP feature-extraction pipeline (FFT spectral features — centroid, noise/voicing/tilt ratios, flatness, rolloff, transient onset — plus an NSDF-based autocorrelation pitch tracker) runs as a compiled WebAssembly engine inside an AudioWorkletProcessor, with a transparent pure-JS fallback when WASM is unavailable.
- WASM AudioWorklet DSP with pure-JS fallback
- FFT spectral features + NSDF autocorrelation pitch tracker
- Per-frame feature vectors drive the rhythm game
- Energy/transient gating and rejection-confidence logic
In-Browser Trainable Classifier
An in-browser RBF-SVM (one-vs-rest) ensemble learns to tell your sounds apart from silence and each other, trained via guided calibration flows (press-and-hold for sustained sounds, a rhythmic panel for plosives).
- RBF-SVM (one-vs-rest) ensemble classifier
- Guided calibration for sustained and percussive sounds
- Custom sound library (percussive/sustained, pitch-invariant/discriminating)
- Export/import your whole library as JSON
Privacy-First, Zero-Framework PWA
No audio ever leaves the device — analysis is entirely client-side and training/calibration data persists locally in IndexedDB, with DSP-config fingerprinting to detect feature-space drift and discard stale corpora. Built with a minimal reactive Signal<T> store and plain Custom Elements.
- No audio leaves the device; training data in IndexedDB
- DSP-config fingerprinting detects feature-space drift
- Zero framework: TypeScript, Signal<T> store, Custom Elements
- Installable, service-worker-precached PWA
Getting Started
No installation required! Visit the live PWA:
https://vp.cognotik.com/
Grant microphone access, then run Quick Calibrate or head to Settings to train sounds. Installable as a PWA for offline use.
Quick Example
Getting Started
- Visit https://vp.cognotik.com/ and grant microphone access
- Run Quick Calibrate (mic gain → pop calibration → sustained-sound calibration)
- Or open Settings to train sounds and tune DSP parameters
- Hit Play and make mouth sounds in time with the rhythm rail
Built-In Sounds
- hiss — sustained, pitch-invariant
- airy — sustained
- pop — percussive (onset-scored)
- hum — sustained, pitch-discriminating
Custom Sounds
- Define your own labeled sounds (icon, label, percussive vs. sustained, pitch-invariant vs. pitch-discriminating)
- Export/import your whole library as JSON
Technical Details
Technologies
Requirements
- Modern web browser with Web Audio and AudioWorklet support
- Microphone access
- No installation or account required
- Offline-capable after first visit (PWA)
Interested in This Project?
This is a proprietary project. For more information or collaboration opportunities, please get in touch.