System overview

The system takes some training songs and creates an output vector of features that characterize the signal.A Gaussian mixture model (GMM) is trained to identify patterns and predict an output instrument classification given a set offeatures.

Each digitized signal was windowed into smaller chunks for feature processing. In training, features werecalculated for each window and concatenated into a single vector to be fed into the GMM for training. In testing, features werecalculated for each window and fed into the GMM for classification. If multiple notes were to be detected, we recurred on the samewindow until we found the maximum number of notes or until a note could no longer be detected (as evaluated using a cutoff thresholdfor what constitutes silence).

From a user standpoint, the user must input a set of training songs, which includes a wav file and the instrumentthat produced the sound at specific times. Once the system is trained, the user can then input a new song, and our algorithm willoutput the song in “piano roll” format, i.e. the pitch and instrument of notes plotted over time.