MelodyDBEngine, which indexed millions of songs’ melody, has been powering Xiaomi, KKBOX and AWA‘s Query By Singing/Humming feature for several years, and its features include: the ability to index music by melodies, search by singing or humming as well as extract melodies from a music files including human singing voice inside of it. This API can be integrated with any music search apps to allow users to search songs by singing or humming a piece of melody.

Today we would like to introduce a new use case based on MelodyDBEngine here: to recognize the cover version of a song on Youtube to know who is the original singer or songwriter in future. To showcase this feature, we created a demo page that allows users to recognize the song name from Youtube by the melody in nearly real-time. Please take a look and play with it and let us know what you think; we would love your feedback!

Using this cover song recognition demo you can simply provide a URL to track the song name of the youtube song, like https://www.youtube.com/watch?v=4bmUFRxNEIg 

Then input the youtube URL to the blank, and click the Search button:

The returned JSON looks like the following, and the score is the confidence of the recognized result.

[
    {
        "score": 0.88888888888889,
        "acr_id": "033466eff9b509b387c22571de08cdc9",
        "title": "despacito (remix)"
    },
    {
        "score": 0.88888888888889,
        "acr_id": "3bdd818903f30bf5e22c272af81abd29",
        "title": "despacito"
    },
    {
        "score": 0.85714285714286,
        "acr_id": "c9c634770ee5f795e8aee710a5d24cfc",
        "title": "despacito (bootlegs) (cutmore amice vocal re edit)"
    },
    {
        "score": 0.75,
        "acr_id": "b3da0027311747de7aeac09986a4400c",
        "title": "sarah - tout doucement - despacito french"
    },
    {
        "score": 0.75,
        "acr_id": "840ebb817460c8f3cb860b65d5677443",
        "title": "despacito (versión urbana\/sky)"
    },
    {
        "score": 0.75,
        "acr_id": "0ea8906df4700ee06a9a6e6b1fc5e9bd",
        "title": "despacito ( tout doucement french version )"
    },
    {
        "score": 0.66666666666667,
        "acr_id": "4d01630031a61a5e88240e5070fce22c",
        "title": "despacito mash up-connor maynard & pixie lott"
    },
    {
        "score": 0.5,
        "acr_id": "ccedb1821245a675650705110e158eec",
        "title": "despacito (versión portugués)"
    },
    {
        "score": 0.5,
        "acr_id": "27ece89de5bb8bdb39de5ba82d2e78cd",
        "title": "despacito low key"
    },
    {
        "score": 0.5,
        "acr_id": "b33e39fe55ba63d99d87298617f26fe5",
        "title": "frozen"
    }
]

Please note that the current recognition process is a little bit slow, because we have to:

1. Download the whole YouTube video, and
2. Then do a frame-to-frame FFT to calculate the main melody line of the song
3. Compare the extracted melody to our MelodyDB in the server-side to get the recognized result.

Duration testing, we also find that the YouTube’s cover version of a song sometimes contains background music that could be identified by ACRCloud’s Audio Fingerprinting Engine (AFP), so we listed 3 configurations in the demo page:

• Audio Fingerprinting & Cover Song Detection, to search AFP database and Melody Database in parallel
• Cover Song Detection Only
• Audio Fingerprinting Only

 

Feel free to have a try.