Codec capers
The inaudible mystery of the missing bits
Brian Dipert, Contributing Editor

Ultimately, the ability of a codec to deliver pristine sound or to minimize file size should be more important than the hype surrounding the technologies. And while few need to understand the intricacies of the algorithms, knowing their capabilities can not only enhance your own personal digital-audio experience, but also lead convergence teams to optimal product specs.

In this introduction, we'll wade quickly into the subject at hand. Should you want some background, both CommVerge and sister publication EDN have covered the topic extensively (see sidebar, "Codec crib notes"). In addition, the sidebar "Compression concepts" explains the fundamentals of data compression.

While I've written quite a bit about digital audio in the last year, my research has been based mostly on second-hand anecdotes, paper specifications, and marketing collateral. An engineer to the core, I always prefer to come to my own conclusions. So I jumped at the chance to do this digital-audio codec project. Only partway through, I've already uncovered some surprising conclusions.

A codec that reduces the size of digitized music is a necessity for storage of such music on low-cost music players and PCs. An uncompressed, dual-channel, 16-bit, 44.1-kHz-sampled audio stream gobbles up nearly 11 Mbytes of storage space per minute of music.

Codecs fall into two basic types—lossy and lossless. Lossless codecs rely strictly on data-compression techniques to reduce file size, ensuring that the decode process will yield an exact copy of the original file. Lossy algorithms discard parts of the audio information, presumably in a way the human ear can't detect, to reduce file size more dramatically.

When the only thing the world seems to want to talk about is MP3 and other lossy codecs, why should you care about lossless compression? Well, all high-end digital-audio products, including the emerging DVD-Audio and Super Audio CD formats, rely on lossless techniques. In addition, audiophiles might want to archive recordings of live performances or digitize dusty LPs. Lossless techniques provide the highest-possible quality and can cut file size by a third or half—nothing to sneeze at.

With the burgeoning possibilities of the convergence era, other lossless characteristics deserve mention. Remember that once you've lossy-converted a sound clip, there's no way to recover the discarded information. And just as with repeated dubbing of an analog audio cassette, each time you modify and re-save an MP3 or other lossy-compression audio file, or transcode it from one lossy format to another, the audio quality further degrades.

You'll need to lossy-compress audio in order to listen to it on a portable player, stream it to Internet listeners, or digitally broadcast it over the air. And if your audio device can directly read and decode MP3 or other lossy-codec format files burned on a CD-R, you've now got the ability to store up to a few dozen hours of music on one disc.

Goals

Putting generalities behind us, what are some of my specific objectives? For the lossless compression algorithms (see table, "Lossless algorithms"), I wanted to measure and compare several key parameters. First, I wanted to determine how long each algorithm took to both encode (compress) WAV files, and to decode (decompress) back to a WAV. I also wanted to contrast the compressed file sizes to the originals to determine each algorithm's compression efficiency.

My goals are quite different for the lossy algorithms (see table, "Lossy algorithms"). Compressed file sizes aren't meaningful as a comparison point for lossy algorithms. A file compressed to a specific bit rate will be equal in size regardless of the codec used. Of course, some companies, such as Microsoft, claim that their codecs afford better audio quality at a given bit rate than do competitors at twice the bit rate. Thus my goals for lossy codecs will be testing for encoder characteristics that can affect sound quality, and determining the time it takes a codec to encode to a specific bit rate.

Needless to say, audio quality is extremely subjective. Recently a neighbor of mine swore that he was unable to hear any difference between an original audio CD, a 128-kbit/sec MP3 stream, a 64-kbit/sec WMA (Windows Media Audio) stream, and a 16-kbit/sec RealAudio stream, all heard through his PC's cheap speakers. When I brought him over to my house and played the same files on my PC's higher-end speakers, he immediately understood the need for those files that "took so long to download." I'll generally resist the urge, therefore, to make quality comments on various codecs, though if one sounds particularly good or bad to my less-than-golden ears, I'll note it. And my tests will certainly allow you to draw some conclusions in terms of quality.

All my results are highly dependent on the characteristics of the source material. Critical factors include the total frequency range represented, the amount of sample-to-sample frequency variation, whether the material is monophonic or stereo, and (if stereo) how much the channels differ in frequency, amplitude, and phase. To balance the tests, I picked quite a variety of genres from my diverse music collection (OK, I admit it's light on country) to throw at the codecs (see table, "Music samples"). I specifically chose not only modern, digitally captured music but also older analog- and live-recorded tracks.

To convert most of the audio tracks to WAV files, I used an enhanced version of Windows 98's CDFX.VXD driver. I also chose a Beastie Boys hybrid CD that contains both audio tracks and PC software. That disc was unusable with CDFX.VXD, so I extracted the desired song using the WAV conversion utility built into MusicMatch Jukebox.

Some codec vendors have been less than fair in their own benchmarking. For example, lossy codec vendors have been known to feed their own codec low-fidelity mono sound clips while giving a competitive codec a full-fidelity stereo clip—a much more challenging task. Note that all the music tracks in my test suite are stereo.

To fully understand the codecs' capabilities and gain some insight into how they work, it's important to use both real music clips and synthetic test tones. I generated a series of test files using Syntrillium Software's Cool Edit Pro. The files include white noise, pink noise, and a 30-second tone pattern.

White noise is a random audio pattern spanning the entire frequency spectrum, with all frequencies represented in equal proportions. Pink noise's sample proportions, in contrast, follow a 1/frequency pattern. In other words, pink noise contains far more low-frequency components than high-frequency ones.

For both the pink- and white-noise files, I generated stereo tones with equal-intensity channels and with different-intensity channels. Equal-intensity noise channels, converted to a frequency-domain display via a spectrum analyzer, enable me to identify any low-, band- or high-pass filtering done by a lossy codec. Differing-intensity channels, among other things, identify whether or not the encoder is simplifying its task by converting stereo to mono within certain frequency ranges.

My 30-second pattern of frequencies has the two channels out of phase. The aim here is to detect the use of joint or intensity stereo compression techniques. In other words, the codec could again minimize its workload by ignoring phase differences between channels within certain frequency ranges, while accurately preserving differences in amplitude.

Finally, to look for a specific lossy-compression artifact called echo, I used the sharp-transition sound file created by striking a triangle, provided by digital audio guru Arnold Krueger (see www.pcabx.com). For more details on all these test files, see table, "Test tones."

Although I created the noise and test tones in Cool Edit Pro, I also used Sonic Foundry's Sound Forge to run some of the lossy decode tests, because it supports more codec formats. For the initial spectrum-analyzer images of MP3 test results (Figure 1a shows an example), I returned to Cool Edit Pro. Cool Edit Pro enabled me to more tightly zoom into the waveform and also included a more robust and informative spectrum analyzer. To see the full set of spectrum-analyzer images, see sidebar, "Visual evidence."

Unfortunately, no single package can decode all lossy formats. In some cases I will have to use a decoder provided by the codec vendor and output the uncompressed bits to a WAV file, which I can in turn input to Cool Edit Pro or Sound Forge. I'm working to ensure that I accurately capture the decoded bits into lossless WAV files.

When you peruse the results, keep in mind that my PC has a fast and full-featured microprocessor. If a given format's encoders and decoders take advantage of the integer and floating-point SIMD (single instruction, multiple data) instructions of my Pentium III, my PC might run them much faster than their non-enhanced counterparts. The differences could be less glaring on a more conventional CPU (for a detailed look at my PC configuration, see table, "Test system").

Lossless results

Even though I've only just begun, I've already gotten some interesting results. I'll highlight some findings here; for the full gamut, download the table, "Lossless results," which is available as a PDF file and as a Microsoft Excel spreadsheet.

To kick off the lossless tests, I decided to run each of the WAV files through the ubiquitous WinZip program. I ran tests at the program's "fastest speed" setting and its "highest compression" configuration. I'd heard that the compression algorithms employed by generic file-compression utilities such as PKZip, WinRAR, and WinZIP were less than optimal for low-entropy multimedia files, and my results certainly bear this out.

Frankly, I was surprised by just how poor WinZip's compression ratios turned out, as compared to the other programs. Generally, WinZip yielded files 80 to 90 percent as big as the originals, while the audio-specific codecs came in at 40 to 60 percent. I was also surprised that WinZip's "highest compression" option, though it took noticeably longer than the "fastest speed" setting to compress large files, didn't achieve proportionally better results.

WinRAR, a peer of WinZip, is unique among generic file-compression programs in offering a multimedia-optimized compression option. This multimedia mode delivered significantly better results than WinZip. However, on average, WinRAR took roughly twice as long to compress a WAV file than did WinZip at even its slowest setting.

As one might expect, the audio-optimized algorithms performed better. But surprisingly, MUSICompress, Shorten, and WavPack all turned in similar results. This seems odd because, according to their documentation, the algorithms vary widely. MUSICompress, for example, is touted as being a lean-and-mean, integer-only routine employing a relatively simple predictive algorithm. Shorten, by contrast, is a more elaborate and configurable floating-point scheme. In fairness, I must admit that I didn't experiment beyond Shorten's default settings.

The compression ratios varied widely with different types of music. Classical music compressed down to less than half the original file size. Modern music, such as rap and techno, proved nearly as random and difficult to compress as pink and white noise (perhaps our parents were right, after all, when they claimed that rock-and-roll was nothing but noise).

I also found that MUSICompress couldn't handle the Beastie Boys' WAV file that I generated using MusicMatch Jukebox. I've heard that some packages produce less-than-100-percent-standard WAV files, which other programs sometimes find difficult to read and play. I believe this scenario occurred here.

Lossy limbo

I've just started running tests on the lossy codecs. The results will be published on this project's home page, with highlights printed in our December issue. In preliminary experiments, however, I've already made some suprising findings based on my experience with the Fraunhofer-developed MP3 codec built into Sound Forge.

For starters, I strongly suspected the Fraunhofer and other encoders would reduce the bit rate by chopping off the upper end of the audio frequency spectrum—which humans can't really hear anyway. And, as Figure 1 shows (see sidebar, "Visual evidence"), the Fraunhofer codec certainly employs this drastic yet effective measure. Both 64-kbit/sec MP3 files severely attenuated the signal beyond 10 kHz.

Next, I took a look at the MP3-converted versions of the pink-noise test tone with unequal left and right channel intensities. I was looking for stereo-to-mono conversion done by the encoder at low or high frequencies. A lossy codec might perform such a conversion to reduce the randomness of the data and simplify the encoding task. A comparison of original and MP3 spectrum-analyzer plots would reveal use of such a shortcut, but I found no evidence that the Fraunhofer encoder used this technique (see Figure 2 and explanation in the sidebar, "Visual evidence").

I also suspect that many lossy codecs might ignore phase differences between channels. But I didn't find any evidence of high-frequency, channel-to-channel phase-difference collapse (see Figure 3 and explanation in the sidebar, "Visual evidence"). I suspect that I simply gave the encoder too easy of a task, with the two channels identical except for their phase and with long sequences of single-frequency samples.

Phase alteration, unfortunately, is very difficult to observe with the more complex multi-frequency patterns that more closely mimic real music. Similarly, construction of a test-tone sequence that will reveal the presence of encoder-generated frequency and temporal masking is also challenging, though I'll strive to achieve this goal.

With the spectrum analyzer, I did find visual evidence of echo, a common side effect of the time-to-frequency transform at the heart of all lossy algorithms. For example, a low-amplitude noise pattern preceded the actual start of the sound generated by the triangle crash (see Figure 4 and explanation in the sidebar, "Visual evidence"). The MP3-compressed version of the triangle sounds "duller" to my ears than the original. And at moderate playback volumes I can even hear the echo-created colored noise burst.

Future plans

Since my lossless compression results didn't uncover dramatic differences, I don't plan to evaluate any of the other lossless codecs. My next step is to run a series of lossy encoder/decoder combinations through the test suite. I'll be using my real sound clips (see table, "Music samples") to determine compression speed and where possible, best-case decompression speed. I'll also use the test tones and the waveform and spectrum analysis displays of Cool Edit Pro to identify the presence of various lossy compression techniques and uncover audible compression artifacts.

As for MP3, I'd like to test drive multiple encoders to see if their results differ. Other high-priority codecs include AAC, Microsoft's Windows Media Audio, RealNetworks' RealAudio, TwinVQ, and Sony's ATRAC.

I'd appreciate your feedback and suggestions as I proceed. Keep an eye out for updates in our print edition and on this project's home page, where I'll post my test tones, tables and graphics, interim test results, and some surprises.

Author information

Contributing Editor Brian Dipert is a technical editor for EDN (www.ednmag.com). As a child, he drove his parents to despair by disassembling his toys—as well as the grown-ups' toys—to figure out how they worked. You can reach Brian at bdipert@pacbell.net or his personal Web page.