Winner of five television Emmys, Dennis Baxter, a broadcast sound designer, author, columnist, lecturer and critic within the sports broadcast industry gives his viewpoint on the next generation of audio technologies.
As a broadcast sound designer, I've spent my career trying to figure out how to achieve the perfect live broadcast sound mix and distribute it to the consumer. But the fact is, that no matter how good the mix is, it’s going to sound different from one home viewer to the next.
Today it’s not just the disparate home viewing environments that are the problem, it’s the different viewing platforms as well. Let’s face it, when it comes to 21st century broadcasting, we live in a Xanadu of consumer choice. Broadcasters are facing stiff competition in retaining viewing audiences who are turning to alternative content sources. And viewers today are choosing to consume content on many different platforms – from traditional televisions to tablets to smartphones – many of which offer a more convenient, interactive and personalized experience. Controlling the broadcast sound reproduction quality across so many platforms has been impossible.
But I know there is hope. I have watched an organization explore these boundaries and design sound applications specifically for the anomalies and diversity of broadcasting. I was aware of Fraunhofer from their research and development of the revolutionary MP3 technology, AAC audio coding and MPEG-2, AVC and HEVC video coding, but recently I had the chance to explore their evolutionary system for the broadcast audio experience. Built upon the new MPEG-H standard which the international organization had made many contributions to, Fraunhofer’s interactive audio architecture for broadcasting adapts sound to fit the platform. It is based upon two key features: immersive sound from above that creates a multidimensional audio space around the listener, and object-based sound composed of individual sound components that the listener can choose to turn up or tune out.
“Interactive, immersive and easily implemented, Fraunhofer’s audio system technology is designed for the superior delivery of content while providing a differentiating edge to broadcasters and enhancing the sound experience of the consumer regardless of platform,” Robert Bleidt, General Manager of the Audio and Multimedia Division of Fraunhofer USA Digital Media Technologies explained. “Our system will offer interactivity using MPEG-H’s object coding, which allows viewers to adjust the sound mix to their own preferences, for example, boosting hard to understand dialogue or creating a ‘home-team’ mix of sports broadcasts. The Fraunhofer system is immersive in that it offers a cinema-like realism through additional front and rear height speaker channels that create a truly realistic sound envelope for the user. 3D sound in cinemas is a trend that the broadcasters may need to maintain competitive parity with.”
What Fraunhofer has figured out is a good sounding bit-rate reduction codec that gives the audio producer additional full-fidelity controllable audio channels with which to design a more engaging entertainment experience. Let’s take a closer look at how this evolution in sound might affect the individual consumer. The handheld tablet screen could become a control board where the sound can be easily optimized for the consumer’s viewing environment as well as their personal preferences. Consumers would be able to contour the sound for the kitchen, bedroom, living room, home theatre, and even their ear buds. The sound control can vary with the user and could be as advanced and interactive as desired or significantly simplified by using preset buttons which can be preprogrammed to do basic functions like boost the dialog or limit the dynamic range. For example, in a noisy environment when an individual user might need to hear the dialog above the background noise, he or she simply clicks Preset Dialog Boost. Interestingly, presets can offer a creative zone for alternative mixes or objects, all of which can be controlled with parameters determined by the sound designer or show producers. As for the less sophisticated user, all this flexibility comes with a reset button that would default to the production sound mix.
Over the last few decades, the visual aspect of broadcasting has been fully explored, from black and white to color, from standard definition to high definition. This is a chance for the television producer to fully explore the experiential value of the sound. To the sound engineer, it presents the opportunity to develop new production workflow practices and offers creative possibilities in exploring the depth of sounds that could be used to heighten the sound experience. For everyone involved, it provides the professional opportunity to evolve and fully embrace the creative possibilities that this personal sonic technology creates. And for the savvy broadcaster who understands this challenge and delivers the amplified consumer sound experience, it provides an opportunity to differentiate their programming and retain viewer loyalty.
Sports broadcasting is a perfect application for the personal sonic experience and technology. In the past few months, demonstration mixes and listening tests of the Fraunhofer sound architecture were conducted using audio from motorsports, extreme skateboarding and snowboarding competitions. With the goal of creating heightened sound realism of the immersive experience of the half pipe and big air, the illusion of vertical sound was created using the upper audio channels.
The result? When the viewers see the athlete on the snowboard go airborne, they also hear the sound of the athlete over their head as well as horizontally on their left and right. The outcome is a new sense of immediacy and reality for the consumer and a new opportunity for broadcasters and networks to broaden their consumer base … as well as a sound evolution.
I am involved in live sports broadcasting and can readily see how the personal sonic technology that Fraunhofer is developing could enhance the consumer experience. Motorsports come to mind because of the vast and rich sound sources available at the races. When viewers are watching and hearing vehicles racing at 200 mph, they would be able to mix in a favorite driver in his car conversing over the crew radio, some track sounds, and enjoy it all in a variety of room sizes and speaker modes with touch of a screen.
Consumer listening and viewing habits are another nightmare for broadcast distribution because reference quality reproduction requires adapting the sound to fit the playback platform. “Given the plethora of speaker configurations in the home environment,” Robert explained, “improved rendering technologies in our sound architecture offer the ability to play any content format on any speaker configuration with a perceptibly improved user experience.”
Finally, as far as implementing this new sound architecture, Robert said, “The new MPEG-H audio system is easily implemented by both broadcasters and consumers. It is Internet-ready for a great listening experience on every device. Building on our experience developing HE-AAC, the native surround audio codec of iOS and Android, our system will also offer DASH support for stutter-free streaming and audio I-frames for easy DASH bitstream switching and splicing for ad insertion. It includes a multi-platform loudness control to provide a tailored experience for a viewer’s device and listening environment. On the production side, it also includes the HOA technology developed by our partners Qualcomm and Technicolor for efficient 3D sound capture and representation.
“The system is backward-compatible with the systems and practices used today for AC-3 or HE-AAC surround sound broadcasting and offers a staged approach to implementing new features,” Robert explained. “We envision four stages:
- Transmission of today’s surround sound with one half the bit rate due to improved coding efficiency
- Addition of interactive objects
- Addition of 3D sound with height channels
- Addition of dynamic audio objects panned to track video action.
Decoders in the system will offer support for all four stages from the beginning, allowing broadcasters to use these features when their facilities and content support them.”
From my perspective as a sound designer, the future of broadcast sound may well be the sound of Fraunhofer’s technology at work from the broadcaster to the consumer.
Fraunhofer’s new television audio architecture builds upon fifteen years of experience providing surround sound using the AAC codec family around the world. Franhofer designed the first AAC encoding hardware for Japanese HDTV in 1998 and AAC 5.1 decoding is supported in all Japanese TVs, AVRs, and soundbars. AAC is also the current codec of NHK’s new 22.2 channel audio system for Super HiVision broadcasting. The work in Japan led to the adoption of HE-AAC Multichannel as the surround codec of the ISDB-based SBTVD system used in many South American countries. Today, Brazilian broadcasters, including TV Globo, use HE-AAC to provide 5.1 surround sound to their viewers. In addition, HE-ACC is also used for surround sound in many countries using the DVB system and the AAC family has been on the air since 2009 providing surround sound for the BBC’s HD service in the UK.