This is a guest post by Ronan Higgins – Founder of TVadSync, the leading provider of TV-to-mobile ad retargeting for agencies and marketers in the US.
Automatic content recognition technology, also known as “ACR”, has advanced significantly in the past couple of years; and the uses of ACR have diversified as quickly.
ACR enables computers to recognize media such as video, images and audio. The application of ACR technology most likely to be familiar to consumers is from music recognition apps like Shazam and SoundHound. These apps listen through the smartphone microphone and can recognize a specific song within seconds, sometime less than a second. To the uninitiated, the user experience of their phone being able to identify a song from millions of songs in the galactic jukebox is magical.
However, ACR comes in various flavors, each with advantages and drawbacks for a particular use case scenario.
On a technical level there are differences in how the recognition is achieved. At its foundation video recognition must be able to recognize a video stream and audio recognition must be able to recognize and audio stream. But even within video and audio recognition there are many ways to skin a cat, both algorithmically and in terms of processor load cost.
Shazam for example uses algorithms to analyze the spectral qualities of audio, from frequency maps to beat recognition. SoundHound has similar, but also has melody recognition that enables it to recognize a tune from humming or whistling.
In video recognition there are various methods used across products, from whole-frame analysis, keyframe analysis and selected pixel analysis to computer vision (CV) pattern, feature and color analysis. Each has benefits in terms of accuracy and speed of recognition and drawbacks in terms of processing power requirement.
One of the top, real-world applications of ACR technology is in advertising — for measuring viewership or exposure to content and commercials; but also for targeting advertising to exposed audiences. ACR offers the opportunity to leap-frog traditional panel-based audience measurement by automating data collection and increasing the scale of data collection by orders of magnitude.
In this regard three main applications have built significant traction in the advertising sector: smartphone app ACR, ACR-powered live broadcast monitoring and smart TV ACR.
Smartphone apps, powered by audio recognition have the ability to delight users by bringing enhanced content based on music, television and movie recognition app, for example, built a TV loyalty program that rewards viewers for watching television by leveraging ACR to validate that the user is watching a specific show or channel. IntoNow app, which was acquired by Yahoo!, enabled users to participate in conversations around the television shows they were watching.
One of the advantages of building the recognition into an app is that it allows advertisers to build anonymous profiles of an individual user’s entertainment tastes, where established audience measurement systems are generally tied to a household’s aggregate entertainment consumption. This is powerful for individual-granular analysis and targeting of relevant advertising. For example, a broadcast network may want to target audiences that have shown a propensity to watch a specific genre of drama in order to promote a soon-to-be-aired new show.
However, in-app content recognition has several drawbacks. The app is only able to listen for audio content when it is open. It’s technically possible to enable an app to listen when in background state but on Apple iOS devices this presents a red warning bar at the top of the smartphone screen that negatively affects the app brand as being intrusive. On Android devices there is no red bar, but users tend to be unhappy about background listening, more from a battery drain perspective than privacy perspective.
ACR is also used for broadcast monitoring, both radio and television. The main application is detecting when ads air, especially competitor ads since advertising agencies generally do not have access to their clients’ competitors’ airing schedule. This “as run” airing data is used by media monitoring companies to generate reports and competitive analysis for marketers who need to understand how they are bench-marking in terms of media spend.
It is also used to enhance household granular viewing information such as cable or satellite set top box “click stream” data. MVPDs generally do not know what commercials are played out on the channels their networks carry, but do know which channels individual households are tuned to at any particular moment in time. By merging ad airing data with set top box data its possible to create a profile of TV commercial exposures or impacts both on a household level but also in aggregate. In aggregate the data has value as an alternative to established ratings currencies such as Nielsen. On a household granular level, TV commercial exposure data has value in media investment decisions such as achieving reach and frequency goals, not only on the television set, but across all the screens in a household.
Similarly ACR technology is being built into latest generation smart televisions to enable collection of household-granular viewing information. Building the ACR into the television set has certain advantages over the set top box data analysis of MVPDs, as television viewing in the home can come from alternate input sources such as Roku, Google Chromecast, Apple TV, terrestrial antenna, DVD/Blue-ray, USB storage, network share, DLNA, and the apps built into the smart television set itself.
Smart TV ACR and set top box data has the advantage over in-app ACR of being able to collect viewing data all the time (well, at least when the device is powered on). However, the information is accurate only to household level. The television and set top box know what is being displayed, but not who is watching. With in-app ACR the the viewing data is tied directly to the owner of the device.
TVadSync TV DMP platform
Household-granular data fits in with the established viewership and ratings currencies. However, when it comes to advertisers wanting to measure and control frequency and reach of impacts across screens and individuals, alternative methodologies have to be applied. For example, an advertising campaign may want to target women over the age of 18. Traditionally, in broadcast advertising this would be done by placement of ad media in relevant shows and channels to the target audience. However, armed with data that a specific household has been exposed to their TV commercial, the advertiser may want to sequence follow-on, call-to-action messaging to the smartphones, tablets and laptops in the household. But these devices may be owned by various members of the household, some outside the target audience. Digital advertising does allow the ability to target “impressions” by demographics such as age and gender, but the big data used by programmatic media buying platforms uses statistical models to determine these demographic attributes. Different models will introduce different levels of probabilistic validity to the targeting filter, which can negate the benefits of the fully deterministic TV viewing data that was at the foundation of the entire effort.
Some television and set top box manufacturers have been experimenting with tools to identify who is actually watching the television from the various members of the household (or even guests). These include Bluetooth beacons built into the electronics, facial or gait recognition (which require a camera unlikely to be enabled by the typical consumer), and even radar.
Ultimately the technology has to provide a value to the consumer participation. One could argue that improved quality of advertising based on tastes and intent signals is enough of a value in itself. Consumers are used to being targeted based on online media consumption, interests and intent signals. This is a good thing as people see more relevant advertising. Broadcast advertising can now enter this realm. This is a tectonic shift and it will take time for the public to appreciate the benefits. ACR is at the center of this. Marketers who learn how the various forms of ACR can best be leveraged will reap strong rewards.
