Song Melody Recognition Research

Make your Arduino dance to a little tune (or just pass on the message)

 Someone with a Samsung washing machine wanted to be able to receive the "all done" notification from the other end of their house.  Here's a video sample with some owners interacting with their washes.


 I didn't turn up a straightforward solution that I could easily implement (it's a lot easier to get music converted to drive a MIDI keyboard) there is plenty of reference material that should lea someone else to a solution.

 Considering the usage time saving to code development time, it might be most practical to just put a wireless baby monitor near the washer.

After glancing though some example projects and academic papers, I concluded that my solution would simply use a tempo-measuring-like setup measuring the interval between notes, if the sound impulses matched the tempo of the song I would call it a match (on review, the notes are not as staccato and easy to separate as a musical greeting card)

Using the microphone built in the little $14.60  M5stickC with an ESP32 would provide the needed horsepower to analyze the simple sound from a monophonic player.

M5stickC COVER REMOVED



Alternatively for basic Arduino's, a Chinese sellers' "Sound Detector" board may be triggered by individual notes.
(You can get 5 of these for $2.94 at Aliexpress)




Passing the "Done" message to the homeowner.

  There are a couple of simple tools for making an Anroid cell phone app that communicates to iOT gadgets.   These tools are the way to go for those who are not competent Android developers.

  One is the Blynk IoT platform which gives you a limited inventory budget  for buttons, dials and sliders that you can place on the screen of your custom app. 

  A second simple app maker I just became aware of is Thunkable (there may be significant differences between the Classic and current versions.)
  One demonstration of Thunkable,  using Bluetooth Low Energy (BLE) to communicate with an ESP32 is in the hackster article "ESP32 BLE + Android + Arduino IDE = AWESOME"  which shows you how to make your Android blink the LEDs on the ESP32 board.

Demo Android App Screen (shrunk)


Here are some (mostly obscure) links I ran across in search of a quick solution.


On the implementation of melody recognition on 8-bit and 16-bit microcontrollers  2003
Author(s) Jyh-Shing Roger Jang ; Yung-Sen Jang

Aubio.org    A code library for audio and music analysis

Essentia
Github C++ library for audio and music analysis, description and synthesis, including Python bindings

MusicTechFest.net  A powerful collection of music processsing tools.

Simplest Way for Voice Recognition Project Using C#, Arduino

Voice controlled TV with an Arduino Uno and C#   2012  (Hacking IR remote code too)

 Parsons Code, a simplified representation format of a song: notes are identified as: same - up or down from previous.

WWW 2018 Challenge: Learning to Recognize Musical Genre

Dejavu Audio fingerprinting and recognition in Python 

RE Cell phone App Shazam:
How does Shazam work? Music Recognition Algorithms, Fingerprinting, and Processing

Step by step: Raspberry Pi offline voice recognition with SOPARE


Open source audio pattern recognition (finger printing)  with a couple Forum Answers 2011

Finding-Song-Melody-Similarities-Using-a-DNA-String  (paprer citation)

Academic Paper:
Resource-efficient Machine Learning in 2 KB RAM for the Internet of Things





Comments

Post a Comment

Popular posts from this blog

LED Driver Roundup

Image
LEDs being current, rather than voltage operated devices require special driver circuits for optimal performance.  While simple applications like an indicator light that might be run at a fraction of it's rated capability, a series current limiting resistor is sufficient.  High power application running at the edge of the LED's max current rating require active regulation to compensate for the negative temperature coefficients of the LED, (measurement techniques are discussed in this university lab paper .) Low power LEDs can be driven with simple series resistor current control when powered from a regulated power supply. The LED Load Line graph be low helps one select a standard resistor value to get the ballpark current setting with a 5 volt power source.  Find where your LED's color line intersects the desired current, then pick the closest resistance line below that intersection. (This graph is from LEDnique.com , a collection of basic articles on LEDs aimed
Read more

Hooking up the 6 + 1 Mic Array to the MaiX BiT

Image
The Risk-V chip on the MaiX BiT board also contains hardware FFT which should make it a respectable audio analysis board.  Sipeed offers a microphone array board with 7 mics and 12 RGB LEDs. Unfortunately the mic array, which SeeedStudio  indicates is for Dock/Go/Bit, there is no real provisions made for the newer BiT board. Included with the mics is a 10 pin ribbon cable for the Go board, and an unadvertised (not on sales page inventory) 5x2 "dupont" connector cable which directly connects between the back of the mic array (not pictured on the sales page) and the Dock board area with mic connection labels silk screened on the board. Some improvising is needed to hook up the BiT.  I offer 2 different solutions. The first that Seeed could help by throwing in a couple of replacement connector housings. Connection Method #1 (You might have to start with a larger housing and cut it down to size) The Seed OPL type for thes
Read more

Holographic Fans Getting Real Time HDMI Input

Image
 One of the technical leaders in so called 3D Holoographic LED Fans is Dseelab, in Sep 2019 they announced at an event the development of real time HDMI Video Input, it may be quite a while befiue it reaches the market.  Achieving this has been my holy grail, looking into FPGA based video processors and examining cartesian to polar code may no longer be necessary... Here's an interesting demo recently posted: Dseelab a leader in the "3D Holographic LED Fan" industry, has master-slave connections on their units, allowing them to be syncronized in an array, they also offer a programming API connecting though WiFi or with a wired serial connection to a PC.  The API supports remote update from the cloud and random play of 256 different media assets. The only other Holographic Fan maker offering a programming API and arrays for large image display is HYPERVSN of  Ireland.  The newest addition to the DseeLab line is their: 6 Blade 78cm, 960*960 res, 84W Model 
Read more