Why do broadcasters care about the transmission of acoustical watermarks? We care because the recovery of these watermarks, using hardware over which broadcasters have no control, is critical to the success of the radio industry.
When Arbitron decided to use technology to replace paper diaries for determining audience measurement, suddenly the technical component of this ratings system became critical to our industry. Today, there are many sources of audio that get broadcast, and the watermark quantity and density generated by Portable People Meter encoders, now provided by Nielsen Audio, are affected by the source material. When this source material is not pure linear audio, the effects on the ability of the watermarks to be transmitted can be quite real, and when combined with the different types of source materials present today, they can cause problems with regard to acoustical watermark transmission success.
My presentation at the NAB Show Broadcast Engineering and Information Technology Conference on Sunday April 23 will summarize research I have done over the past year to look at the effects of bitrate reduction (BRR) schemes on the ability of acoustical watermarks to be transported through normal radio signal paths and ultimately to the PPM appliance for proper decoding and subsequent ratings credit.
A THIRD WAY
To my knowledge there are only a few ways to measure the success of watermark transmissions over the air reliably.
One method is with access to a PPM appliance that a panelist would wear and the data that is recorded therein. Unfortunately for the broadcasters, this data is not generally available in its raw form to use as measurements due to the control and ownership of the raw data by the company engaged in collecting the data.
A second method is to access data from an official PPM confidence monitor issued by the company collecting the data. The monitor gives some indication of the quality and quantity of codes received, but the granularity needed to make accurate determinations of PPM watermark throughput was not readily available with the newest version of the confidence monitors that Nielsen routinely supplies to radio stations and with which I have had the opportunity to get familiar.
Another method is an emerging product by the name of TVC-15. A working prototype was made available to me for this research, and the product has since been released by The Telos Alliance’s 25-Seven Systems.
The TVC-15 is a device that accepts (analog) stereo audio from any source, and contains technology to detect the presence of and extract watermark quality and quantity. It also provides high-resolution data that can be used to evaluate the success of the acoustical watermark transmission.
Keep in mind that not all source material is delivered to the radio station in a linear format; not all broadcast chains are linear (especially STLs); HD Radio is not linear; and we do not fully understand the effects of audio compression as it relates to acoustical watermarks. This could have a significant impact on ratings and revenue derived from PPM technology.
MAKING THE MEASUREMENTS
I selected the TVC-15 because it was the only method available that would supply the required data with sufficiently high time resolution to make meaningful measurements of watermark transmission possible.
Specifically, the TVC-15 came equipped with an Ethernet port supplying 400 millisecond data updates, as well as confidence level values (in the range of 0.00 to 1.00), and reset interval counts resetting each time a valid complete symbol was received and properly decoded.
I designed a program to capture the TVC-15 data in real time and then create text files that can be graphed. It also counts the reset intervals per minute or per 10-second interval depending on the duration of the program material being evaluated. The program also averages the confidence level per minute or 10-minute interval. In general, I employed one-minute averaging for long-form analysis, and 10-second averaging for songs.
CORRELATION WITH NIELSEN RESULTS
To legitimize the results, I decided to see if the values from the TVC-15 would correlate with real-world performance of the throughput of the watermarks.
The only way this could be accomplished was to obtain from Nielsen the actual meter counts obtained from real-world panelist listeners — Nielsen routinely makes these raw meter counts available to subscribers — and to match that information up with the TVC-15 data. A direct correlation between these data sets would give me the confidence to know that the TVC-15 data was tracking with what was going on in the real world.
Realizing this, I made several measurements with different program formats over a one-hour period. All these measurements compared raw meter counts obtained from Nielsen with message success rates as reported by the TVC-15. This work was done with the cooperation of several stations in Boston along with the lab work I performed. In all cases the actual real-world meter count numbers tracked proportionally with the TVC-15 reported confidence levels.
Another test to prove the usefulness of the TVC was to look at the repeatability of the data reported by the TVC. Using the same program material, I ran the same test six times with identical source material, then graphed the results. This information is part of my presentation at the NAB Show, and shows very good repeatability. After an extensive testing period, I was convinced the TVC-15 was for real and could help me gauge the watermark transmission success from a quantitative perspective.
I wanted to look at watermark success rates with different audio compression rates both before and after PPM encoding was applied. For example: Does a highly compressed MP3 encode any worse than the same audio in linear form that gets compressed after PPM encoding, as would be the case with a compressed studio-to-transmitter link or HD Radio transmission?
Fig. 1: A test setup used for determining the watermark transmission success rate for files that were compressed before encoding.
Click to EnlargeFig. 2: This particular voice sample seemed to transmit watermarks most effectively at 32 kbps, and linear audio trailed far behind.
Click to Enlarge
Fig. 1 shows one of the test setups used for determining the watermark transmission success rate for files that were compressed before encoding. A computer containing the files played audio to a PPM encoder, and then into the TVC-15. The TVC then measured the success rate and reported it to a second PC running custom software that was designed to capture and tabulate the TVC’s real-time data. The data subsequently was written to a text file and graphed to produce comparison plots.
Fig. 2 is an unusual example of a surprise I found, showing that this particular voice sample seemed to transmit watermarks most effectively at 32 kbps, and linear audio trailed far behind.
I have much more data to share at the NAB BEITC, and a companion paper will be published in the conference proceedings in case you can’t make it to Las Vegas.
CONCLUSIONS
Bitrate reduction does seem to have an impact on watermark transmission and the ability to detect these watermarks reliably. Most music seems to benefit from a higher bit rate, although 64 kbps does not seem to suffer much compared to linear audio. Voice seems to benefit from MP3 compression, perhaps because of artifacts in the spectral content, especially at 32 kbps.
The TVC-15 has proven to be a unique and valid tool for examining watermark transmission, and the application that I have used it for may just be the tip of the iceberg. Today electronic ratings are such a big part of what we do as radio broadcasters operating in PPM-rated markets, and it is in our best interest to understand the technology as best we can.
These measurements are a moving target, and the more data obtained, the higher the confidence will be with regard to the relevance of the measurements made in this report. Certainly, more study is needed to raise confidence in the results presented in this paper.
The author is director of technical operations for Beasley Media Group’s Boston operation.
For the BEITC session schedule and locations, see www.nabshow.com/education/conferences.
