This is a guest post written by Thomas Lindauer, the developer of Chrome music recognition extension – Beat Finder.

I recently decided to make a Chrome plugin that can identify songs playing in your browser, I’ve discovered a method of developing recognition plugins for Chrome painlessly that I’d like to share.

[youtube https://www.youtube.com/watch?v=umW_KbYwQKU] This demo video shows how Beat Finder works

In this how-to i will explain how i made Beat Finder and how you can make extensions like it. This is not intended for beginners in web development, but more aimed for people with Chrome extension development experience or anyone with intermediate web development skills.

Why is it easy?

Rather than developing your own recognition algorithms and maintaining the usually massive and complicated database architecture, we will use an established and easy to use music recognition service delivered by ACRCloud.

They also deliver Broadcast Monitoring Services, TV Channel Detection Services and more.
Check out their website and get inspired.

ACRCloud has a standardized and easy API to operate, you of course need to register a user and using your API key you send files to their server. Returned from the server is a JSON object with recognition results.

---

How to record what the user wants to identify.

Chrome API to the rescue, there is a Chrome API call you can make that records audio and video from the current active tab.
chrome.tabCapture – Use the chrome.tabCapture API to interact with tab media streams.

I’ve been struggling with this API for a while, in my opinion it is not documented well enough. Let me share its secrets that i’ve discovered.

chrome.tabCapture.capture({

  audio : true,
  video : false

  }, function(stream){

  // continue to play this audio stream
  var audio = new Audio(window.URL.createObjectURL(stream));
  audio.play();

  // create blob
  streamObject = stream;

  mediaRecorder = new MediaStreamRecorder(streamObject);
  mediaRecorder.mimeType = 'audio/webm';

  mediaRecorder.ondataavailable = function (blob) {

    uploadBlob(blob);
    stopCapture();

  };

  mediaRecorder.start(recordTime*1000);

});

In the extension code this would be the call to chrome.tabCapture that does the actual recording from the current active tab.

chrome.tabCapture.capture({

  audio : true,
  video : false

},

In the callback that exposes the tab media stream you first need to continue playing the stream, without this the tab will not play any sound whilst it is being recorded.

}, 

function(stream){

  // continue to play this audio stream
  var audio = new Audio(window.URL.createObjectURL(stream));
  audio.play();

Now for the actual recording of the stream and passing it to ACRCloud.

Use a MediaStreamRecorder object to easily record the stream into a ‘audio/webm’ format or any other format you might need.

  mediaRecorder = new MediaStreamRecorder(streamObject);
  mediaRecorder.mimeType = 'audio/webm';

  mediaRecorder.ondataavailable = function (blob) {

    uploadBlob(blob);
    stopCapture();

  };

  mediaRecorder.start(recordTime*1000);

});

Passing the data to ACRCloud

Pass the file to a server using Ajax and deal with the returned data. This is a basic Ajax upload to a server that has the following server code.

var uploadBlob = function(file){

  /*
    Uploads file to server and waits for file check.
  */

  var fd = new FormData();
  fd.append('fname', 'test');
  fd.append('data', file);
  $.ajax({

    type: 'POST',
    url: 'URL_RECOGNIZE',
    data: fd,
    processData: false,
    contentType: false,
    success:function(data){

      parseRetrievedData(data);

    },
    error:function(error) {

      parseRetrievedData(error);

    }

  });

}

Before you record the current tab i advise to check these values, they will make sure you only record when the tab is active or else you will get errors.

chrome.tabs.query({active:true,currentWindow:true},function(tabArray){

  if (tabArray[0] == null){

    chrome.runtime.sendMessage({order: "lostFocus"});

  }
  else if (tabArray[0].url == null){

    // Lost focus of current tab
    chrome.runtime.sendMessage({order: "lostFocus"});

  }else{

    audioManager.captureStream();

  }

});

Information

The entire Beat Finder project is open source, from server code to the actual plugin. You can check out the GitHub here.

chrome.tabCapture – Use the chrome.tabCapture API to interact with tab media streams.
MediaStreamRecorder – Easily record tabCapture stream into an acceptable format.

ACRCloud server recognition code.

Hope to see many of you develop your own Chrome extensions, to help people identify all kinds of media directly through their browser!