Audio-to-MIDI Conversion Model

Spotify has developed and open-sourced Basic Pitch, a lightweight and accurate machine learning model for converting audio to MIDI. This tool is designed for artists and producers, offering polyphonic and instrument-agnostic transcription with pitch bend detection, and can run faster than real-time. The model utilizes a shallow architecture with a harmonic constant-Q transform, jointly modeling onsets, frames, and multipitch information, resulting in a small footprint (<20MB peak memory, <17K parameters). The project also aims to inspire the development of other lean ML models.