While its name may not be widely known among consumers, virtually everyone has either owned or used products developed by the Fraunhofer Institute for Integrated Circuits (IIS).
Developers of the MP3 audio codec, Fraunhofer IIS is just one part of what is Europe’s largest organization for applied research. Today, Fraunhofer comprises 60 institutes and has a total of around 17,000 employees. Its annual budget is the equivalent of about $2 billion.
Robert Bleidt
Fraunhofer IIS focuses on audio and broadcast technology, among other topics, and has a long history of innovation. It began with the idea of finding a way to send high-fidelity music over telephone lines. Fraunhofer was also the co-developer of the AAC codec and the subsequent High-Efficiency AAC codec, which is now a part of the MPEG-4 audio and video standard. Additionally, Fraunhofer has worked on many of the digital radio broadcasting platforms that exist today.
While attending the NAB Show a few weeks back, I took the opportunity to meet with Matthias Rose, head of marketing communications for Fraunhofer IIS Audio & Multimedia, and Robert Bleidt, the division general manager of Fraunhofer USA Digital Media Technologies. We talked about the history of Fraunhofer and its role in recent audio technology developments, as well as some of their new products introduced or on display at NAB.
Tell me a little bit about the history of Fraunhofer Institutes and the Fraunhofer Institute for Integrated Circuits.
Rose: Fraunhofer is actually Europe’s largest organization for applied research. However, in addition to our focus on research, we also have a strong dedication to bringing the technologies we develop to the market. That’s why we’re here at NAB. This differentiates us from other research organizations, which often do not have such a strong market focus.
For Fraunhofer IIS, the most important products we have developed in the last 20 years are in the field of audio coding and digital broadcast technologies. So when it comes to audio coding, we are the main inventors of the MP3, and we are co-developers of AAC, making us pretty much developers of the most important MPEG [Moving Picture Experts Group, formed by the International Standards Organization to develop standards for audio and video compression] standards that are nowadays used, for example, in broadcasting. They were mainly developed by us in Erlangen [Germany]. We have been doing audio codec development for more than 20 years, currently with a staff of over 120 engineers working on those technologies.
When was Fraunhofer IIS begun?
Rose: Fraunhofer began in 1949. It was founded as a research organization. Fraunhofer today consists of 60 institutes, which are like business divisions. Fraunhofer IIS is one of those business divisions, and is the biggest institute with a total of 750 employees.
Matthias Rose
Fraunhofer IIS started with microprocessor technologies when we were founded in 1985. The first thing that we concentrated on was integrated circuits and circuit design, and then we rapidly went into the audio business, and so from the very beginning we worked on audio technologies. Today, we are in the fourth generation of audio technologies, so MP3 was only the beginning.
Didn’t you work on the European digital broadcasting standard, Eureka 147? That was my first introduction to MPEG and audio codecs.
Rose: Yes, that is right. Part of this work in the 1990s was also the development of audio codecs and our participation in the MPEG standardization. MPEG decided to have three different audio codecs standardized at the same time in MPEG-1. There is one codec with a very low complexity, Layer I, which nowadays is not really used any more, and then Layer II was standardized with more complexity and compression, and finally Layer III as the codec with the highest quality and also the highest complexity. We provided this Layer III together with partners like Thomson and AT&T Bell Laboratories.
We have a very extensive Web page on the MP3 history where we explain it all. The URL is www.mp3history.com. Then in 1997 came AAC, and then High-Efficiency AAC [HE-AAC]. Now we are in the fourth generation of surround codecs like MPEG Surround.
So as you can see, we did quite a lot in the last 20 years, and this is just the audio part. In broadcasting, after Eureka, we at Fraunhofer helped to develop the WorldSpace system for WorldSpace Satellite Radio, including the audio codec. We were also involved in XM Satellite Radio and DAB digital broadcasting in Europe. When it comes to radio broadcasting, you hardly can find a digital broadcast system where we haven’t participated in some way.
Why was Fraunhofer IIS formed in 1985?
Rose: It was founded to advance developments in the rapidly expanding field of microelectronics. One of the first industry projects in 1985 was an intelligent running shoe.
The Fraunhofer IIS acoustic lab was designed for the strict standards of MUSHRA listening tests.
In 1987, the institute formed a research alliance with University of Erlangen-Nurem¬berg within the framework of the European Union-funded Eureka project EU147 for Digital Audio Broadcasting. This marks the beginning of audio codec development at Fraunhofer. Audio codec development was initially started at the University of Erlangen-Nuremberg in the late ’70s with the idea of Professor Dieter Seitzer of transmitting audio over ISDN telephone lines.
Who owns Fraunhofer?
Rose: Fraunhofer is a non-profit organization. More than 70 percent of the Fraunhofer contract research revenue is derived from contracts with industry and from publicly financed research projects. Almost 30 percent is contributed by the German federal and Länder [state] governments in the form of base funding. This is a very unique setup. We are the largest organization like this in Europe, and I’m pretty sure also worldwide.
It’s one reason our customers like to work with Fraunhofer because they can make use of our long experience. Fraun¬hofer is one of the most attractive employers in Germany; that means we really get the best engineers out of university.
Our customers value our services because they know they get the best possible engineering services and they also know that we really care about the quality of the products, especially the audio quality.
What type of work does Fraunhofer do on a day-to-day basis today?
Rose: We at Fraunhofer IIS Audio & Multimedia develop audio and multimedia technologies and license them to the marketplace. This includes licensing of software implementations of audio codecs to end-product manufacturers.
Our customers value our software implementations, as they know that they get the best possible quality due to our extensive listening tests. We have one of the most modern listening and acoustic labs worldwide. It is in a building that was constructed just 2–3 years ago, and it uses a room completely decoupled from the rest of the building. It was designed for the very strict standards of MUSHRA listening tests.
Fraunhofer IIS is the largest division within Fraunhofer Institutes.
Can you share with us a couple of stories about the development of the MP3 audio codec?
Rose: There’s a whole collection of those stories on our MP3 history website.
One early story concerns the efforts of Professor Seitzer to develop his idea of transmitting high-fidelity audio over telephone lines. His first patent application to do so was rejected because the patent lawyers said that it would never be possible to do something like that.
Our institute director, Professor Heinz Gerhäuser, tells a nice story about how they visited the BBC in their early days and they brought all this hardware for some listening tests. But due to a misunderstanding, the prototype audio rig was designed to operate at the wrong operating level and with inverted phase. It was late night at the BBC offices and no one was there anymore, and they were in this hardware lab with everything closed up and staring at a locked parts cabinet. The next morning they were scheduled to present their system for formal listening tests to the BBC.
One of the engineers got the idea to just get a broom and sweep around under the lockers to find any parts that may have dropped, hoping that the cleaning staff had not been all that thorough. They turned up a handful of integrated circuits and found what they needed, sitting on the floor under the cupboards, to design the proper interface and they were able to do the demonstration.
Early work by engineers also included drilling holes in a standard CD player in order to mount up an external digital interface so that high-quality source audio could be obtained for listening tests, as told by Thomas Sporer, currently head of the Department of Acoustics.
Are you working specifically on a codec that might be designed for voice channels? In my experience with low-bitrate codecs, music generally sounds very good, with little audible artifacting, but voice material does exhibit artifacting.
Bleidt: Voice signals are extremely difficult for a number of reasons having to do with the nature of the speech signal. It’s really unlike most music signals in that it has a lot of impulsive transients, which you don’t normally hear so frequently in music. The traditional methods that you would use in a music encoder to deal with transients don’t really work that well for speech. It’s almost exclusively a transient-loaded signal.
Much of the work done on voice codecs has been done by the mobile phone companies and they tend to work at very low bitrates with the goal of highest efficiency for comprehension of content only.
What are some of the latest products to come out of Fraunhofer research activities?
Bleidt: Well let’s start with the Sonnox plug-in that we’re showing at NAB. The concept behind the plug-in was to allow engineers or producers who are mixing audio to actually hear what the end product is going to sound like after the MP3 or AAC encoding is applied.
We saw that as a need in the broadcasting and music recording industries that wasn’t really being addressed, and of course, we had the technology at Fraunhofer to enable this. We decided to partner with Sonnox, the former Sony Oxford digital console company, which now makes plug-ins for digital audio editing software, to bring this real-time plug-in to the marketplace.
What it offers is the ability to encode and decode in real time all of the MP3 codecs and all of the AAC codecs for anyone who is using digital audio software that uses VST plug-in, or the Audio Units plug-in, or the RTAS plug-in that ProTools uses.
The Sonnox plug-in allows those who are mixing audio to hear what the end product is going to sound like after the MP3 or AAC encoding is applied.
That saves a lot of time in audio production because you don’t have to output the finished WAV file from your audio software, encode it and then play it back and realize maybe I need to change this and have to go back. You can do all that online.
The other way it saves time is when you are making multiple types of outputs. Say you want to make different versions, with an MP3 for the archive and an AAC for a podcast, you can make all of those at once. It also supports any of the surround formats, both AAC and MP3 Surround.
Additionally, there are a lot of technical tools in the plug-in. We also realized that people really weren’t evaluating our codecs as they should. People would do very casual listening and then say that they did or did not like a particular codec. As we mentioned earlier about testing to evaluate audio codecs, you really need double-blind testing to do a fair comparison, because we’re all inherently biased. Which content we heard first, which was louder, and other factors can have an effect on our opinion. And if you know which encoder was used, that’s an in-built bias you can’t get around. The plug-in has a built-in tester for this so you can do double-blind listening tests yourself.
Finally, we also included some indicators that can alert you to potentially troublesome situations. One of those indicators is the noise-to-mask ratio for each of the frequency bands of the encoded signal. You can see from looking at that if there’s a chance you could hear an encoding artifact.
Another new announcement that we have for NAB is the integration of MPEG Surround into Orban’s products. So it’s now possible for a station to use the Orban Internet radio encoding package to send out MPEG Surround. They’ve been doing encoding with HE-AAC v2, and that has worked out well with some broadcast properties, and now we’re offering the option to do real surround broadcasting over the Internet at the same bitrate as stereo, 64 kilobits per second.
The idea there is to be able to put together a complete signal chain, and we’ve done some good work on that. At CES we announced a partnership with Texas Instruments to get our MPEG Surround decoder into the next generation of audio/video receivers. Most of the high-end AVR receivers today are starting to have an IP connection built in to them so they can get Internet radio services. The Texas Instruments Software Development Kit is used by many of the manufacturers of AVR equipment.
We can’t name names yet, but it seems that there will be a high-end AVR receiver with MPEG Surround available in the 2012 time frame. We have a prototype here at the booth.
Interviewer Michael LeClair is technical editor of Radio World Engineering Extra and chief engineer for WBUR, Boston.
Comment on this or any other article to [email protected].
