
Describes a device that uses pitch detection built on Arduino. It recognizes the notes played on a musical instrument such as a clarinet. It won 1st place in the Silicon Valley science competition.
Introduction
While playing my clarinet, I realized that it would be fun to hear other instruments playing alongside me. Instruments like guitar, piano or even a choir. It would also be nice if the melodies could be transcribed on paper. All existing solutions to these problems require a bulky computer or a cell phone. I realized that creating this compact device would combine my interest for music with my passion for engineering and math.
This project creates a small, affordable and accurate device that listens to a musical instrument and recognizes the notes played. These notes can then be sent to a synthesizer in the common MIDI format. This allows musician to hear other instruments playing alongside with them, and allows them store their compositions.\(\)
The implementation is in C++ and uses an Arduino UNO, breadboard, microphone and optional display. It displays the music as a piano roll and sends it to an external synthesizer.
Those that want to jump straight to the core, read at least the hardware page and download the code through
Background
Five years ago, I asked my dad “How do computer do math?”. Ever since, we have spent countless hours learning about the semiconductor physics, diode logic, programmable logic and microprocessors. I find it fascinating to learn about physics and engineering. With the help and dedication of my father, I then started programming. He has provided valuable insights and suggestions allowing me grow. While it can be frustrating at times, I also find it very enlightening to build programs with ever increasing complexity.
For this project, I conducted the research and development at home. As always, my father supervised the project and helped me with architecture and code reviews to keep it readable and maintainable. He also provided suggestions to organize the code to make modules reusable and testable. In particular, he helped me when the software appeared to have random crashes. He explained about heap and stack pointers, and suggested to test for at least a 50 bytes headroom after the data structures.
Goal, design criteria and constraints
The project creates a small embedded monophonic music transcription system. Monophonic means one sound, so you can transcribe one note at a time. Therefore you cannot use this device to analyze what a band is playing without each instrument playing separately.
The device samples an analog audio signal, processes it and creates a digital MIDI output. MIDI is a digital annotation for music that signals the beginning and ending of each note.
I set out to adhere to the following design criteria:
- It should detect the correct pitch for notes produced by a B♭ clarinet but also work for other monophonic instruments.
- The beginning and duration of each notes should be identified, regardless of it being followed by a rest, note change or note repetition. This process is called segmentation.
- There should be no noticeable delay between the incoming audio signal and the digital MIDI output.
The cost of the initial prototype, excluding the optional display, should be under $20 and fit on a credit card size PCB. To address these constraints, I chose to build the device around the commonly available Arduino UNO, an open source prototyping platform based on an 8-bit 16 MHz microcontroller with only 2 Kbyte SRAM. Using such a slow microcontroller with limited memory posed interesting challenges but also keeps the cost down. By creating a custom PCB, the final product can be approximately 1 inch in diameter what allows it to be clipped onto an instrument.
Methodology
The common method of detecting a note pitch is by using a fast Fourier transform (FFT). This project shows that similar results can be achieved using autocorrelation. This method has the advantage of requiring only half the amount of memory compared to a FFT. The algorithm incorporates optimizations such using a low sample rate, simplifying the autocorrelation formula, using interpolation and asynchronously sampling the audio signal.
Testing with public audio samples, I found that all clarinet notes from 155.6 Hz (Eb3) to 1568 Hz (G6) are recognized correctly. Using normalization, this range can be extended to 98 Hz (G2) to 1568 Hz (G6) for the wider range of the piano. This small device enhances the experience of playing an instrument.
The source code is shared through GitHub what allows others to learn and improve this product.
Following chapters describe my project in detail. From the hardware schematic to verification methods. I hope you enjoy learning from my experience.
Hello,
I want to know the frequency of the power grid, but harmonic frequencies in the power grid are a huge problem.
Can you send me a code for ‘filtering’ fundamental? (0-150Hz max)
Thanks!
great works !!!
Hello,
First of all, this a great work ! Not only just a sample stuff but very documented, you shoudl be proud !
I’d like to use it for a guitare. Do you think it will work ? Did you test it ?
Finally, did you make a video of it to see how good it works ? :)
Thanks for the sharing and keep having fun making all these things :)
Hello. Bravo for this work …
I am a retired electronic engineer, and I designed a tuner working with FFT on a PC with a professional tool : Labview from National Instrument, which offers huge signal processing libraries. I wanted a very precise instrument to tune the reeds of a vintage italian accordion (Paolo Soprani, 1915). I am also playing bassoon, and then concerned with bass notes. I encountered many difficulties with low frequency cut off of the microphones, giving signals with harmonics far higher than fundamental (up to 20 dB). The lowest note of my accordion is a Bb at 58.27 Hz, on the left hand chords. This is also the lowest note of the basson. Precise measurement need very long sampling of several seconds !
Well, now, I want to buid a little funny gadget which will animate a “snake” poping out of a basket in relation with the recognised notes … I am very interested by your work, but I think i will use a more powerfull processor than the Arduino uno, and then higher sampling frequency and 500 or 1000 samples.