AURALIS (2026)

A local evaluation harness for comparing how audio-capable multimodal language models describe the same musical excerpt. Each run hashes the uploaded audio (WAV, MP3, FLAC, AAC, or OGG), encodes it as base64 input_audio, and dispatches identical system instructions, prompts, and sampling parameters across selected models with provider fallbacks disabled.

Responses, latency, token usage, and normalized musical descriptors are persisted in SQLite via Drizzle ORM and exported as JSON or CSV for blind side-by-side comparison.