Introduction

By our entrepreneur in residence, Johan Vonk

Describes a device that uses pitch detection built on Arduino. It recognizes the notes played on a musical instrument such as a clarinet. It won 1st place in the Silicon Valley science competition.

Introduction

While playing my clarinet, I realized that it would be fun to hear other instruments playing alongside me. Instruments like guitar, piano or even a choir. It would also be nice if the melodies could be transcribed on paper. All existing solutions to these problems require a bulky computer or a cell phone. I realized that creating this compact device would combine my interest for music with my passion for engineering and math.

This project creates a small, affordable and accurate device that listens to a musical instrument and recognizes the notes played. These notes can then be sent to a synthesizer in the common MIDI format. This allows musician to hear other instruments playing alongside with them, and allows them store their compositions.\(\)

The implementation is in C⁺⁺ and uses an Arduino UNO, breadboard, microphone and optional display. It displays the music as a piano roll and sends it to an external synthesizer.

Those that want to jump straight to the core, read at least the hardware page and download the code through

Background

Five years ago, I asked my dad “How do computer do math?”. Ever since, we have spent countless hours learning about the semiconductor physics, diode logic, programmable logic and microprocessors. I find it fascinating to learn about physics and engineering. With the help and dedication of my father, I then started programming. He has provided valuable insights and suggestions allowing me grow. While it can be frustrating at times, I also find it very enlightening to build programs with ever increasing complexity.

For this project, I conducted the research and development at home. As always, my father supervised the project and helped me with architecture and code reviews to keep it readable and maintainable. He also provided suggestions to organize the code to make modules reusable and testable. In particular, he helped me when the software appeared to have random crashes. He explained about heap and stack pointers, and suggested to test for at least a 50 bytes headroom after the data structures.

Goal, design criteria and constraints

The project creates a small embedded monophonic music transcription system. Monophonic means one sound, so you can transcribe one note at a time. Therefore you cannot use this device to analyze what a band is playing without each instrument playing separately.

The device samples an analog audio signal, processes it and creates a digital MIDI output. MIDI is a digital annotation for music that signals the beginning and ending of each note.

I set out to adhere to the following design criteria:

1. It should detect the correct pitch for notes produced by a B♭ clarinet but also work for other monophonic instruments.
2. The beginning and duration of each notes should be identified, regardless of it being followed by a rest, note change or note repetition. This process is called segmentation.
3. There should be no noticeable delay between the incoming audio signal and the digital MIDI output.

The cost of the initial prototype, excluding the optional display, should be under $20 and fit on a credit card size PCB. To address these constraints, I chose to build the device around the commonly available Arduino UNO, an open source prototyping platform based on an 8-bit 16 MHz microcontroller with only 2 Kbyte SRAM. Using such a slow microcontroller with limited memory posed interesting challenges but also keeps the cost down. By creating a custom PCB, the final product can be approximately 1 inch in diameter what allows it to be clipped onto an instrument.

Methodology

The common method of detecting a note pitch is by using a fast Fourier transform (FFT). This project shows that similar results can be achieved using autocorrelation. This method has the advantage of requiring only half the amount of memory compared to a FFT. The algorithm incorporates optimizations such using a low sample rate, simplifying the autocorrelation formula, using interpolation and asynchronously sampling the audio signal.

Testing with public audio samples, I found that all clarinet notes from 155.6 Hz (Eb3) to 1568 Hz (G6) are recognized correctly. Using normalization, this range can be extended to 98 Hz (G2) to 1568 Hz (G6) for the wider range of the piano. This small device enhances the experience of playing an instrument.

The source code is shared through GitHub what allows others to learn and improve this product.

Following chapters describe my project in detail. From the hardware schematic to verification methods. I hope you enjoy learning from my experience.