Vocal Parkour

A browser-based vocal-gesture rhythm game: make mouth sounds (hiss, airy, pop, hum — or record your own) into your microphone, and a fully client-side, real-time audio engine classifies each sound while a rhythm rail scores timing, precision, and consistency. Privacy-first: no audio ever leaves the device. **Play it at vp.cognotik.com →** — grant mic access, run Quick Calibrate, and you're playing in about a minute.

Visit Website

Key Features

Real-Time Client-Side Audio ML

A DSP feature-extraction pipeline (FFT spectral features — centroid, noise/voicing/tilt ratios, flatness, rolloff, transient onset — plus an NSDF-based autocorrelation pitch tracker) runs as a compiled WebAssembly engine inside an AudioWorkletProcessor, with a transparent pure-JS fallback when WASM is unavailable.

WASM AudioWorklet DSP with pure-JS fallback
FFT spectral features + NSDF autocorrelation pitch tracker
Per-frame feature vectors drive the rhythm game
Energy/transient gating and rejection-confidence logic

In-Browser Trainable Classifier

An in-browser RBF-SVM (one-vs-rest) ensemble learns to tell your sounds apart from silence and each other, trained via guided calibration flows (press-and-hold for sustained sounds, a rhythmic panel for plosives).

RBF-SVM (one-vs-rest) ensemble classifier
Guided calibration for sustained and percussive sounds
Custom sound library (percussive/sustained, pitch-invariant/discriminating)
Export/import your whole library as JSON

Privacy-First, Zero-Framework PWA

No audio ever leaves the device — analysis is entirely client-side and training/calibration data persists locally in IndexedDB, with DSP-config fingerprinting to detect feature-space drift and discard stale corpora. Built with a minimal reactive Signal<T> store and plain Custom Elements.

No audio leaves the device; training data in IndexedDB
DSP-config fingerprinting detects feature-space drift
Zero framework: TypeScript, Signal<T> store, Custom Elements
Installable, service-worker-precached PWA

Getting Started

No installation required — it's a browser PWA.

👉 Play now at vp.cognotik.com →

Grant microphone access, then run Quick Calibrate (or head to Settings to train your own sounds). Install it to your home screen for offline play. Your audio never leaves the device.

Quick Example

Getting Started

Open vp.cognotik.com and grant microphone access
Run Quick Calibrate (mic gain → pop calibration → sustained-sound calibration)
Or open Settings to train sounds and tune DSP parameters
Hit Play and make mouth sounds in time with the rhythm rail

Built-In Sounds

hiss — sustained, pitch-invariant
airy — sustained
pop — percussive (onset-scored)
hum — sustained, pitch-discriminating

Custom Sounds

Define your own labeled sounds (icon, label, percussive vs. sustained, pitch-invariant vs. pitch-discriminating)
Export/import your whole library as JSON

👉 Play at vp.cognotik.com · Also try The Arcade of Life and Mathematical Explorations

Technical Details

Technologies

TypeScriptWebAssemblyWeb Audio APIAudioWorkletDSPMachine Learning (RBF-SVM)HTML5 CanvasIndexedDBService WorkersPWA

Requirements

Modern web browser with Web Audio and AudioWorklet support
Microphone access
No installation or account required
Offline-capable after first visit (PWA)

Interested in This Project?

This is a proprietary project. For more information or collaboration opportunities, please get in touch.