Stay in touch…

Blog

Read the latest Bitstream

RSS Feed

LinkedIn

Look for us at LinkedIn

Twitter

Follow us on Twitter

Mix Magazine

This installment of The Bitstream column appeared in the May 2004 issue of Mix Magazine.

The Bitstream

This column takes a second look at lossless codec for digital audio…

A Profound Loss

This month, I’m taking a (second) look at an increasingly viable alternative to both bulky LPCM sound files and lossy compressed but compact audio files. That alternative is a lossless codec, which trims the fat without sacrificing aural satisfaction.

Let’s start with my “viable alternative” descriptor. In some parts of the world, notably Western Europe, the Pacific Rim and Canada, FTTH and FTTC has arrived in urban neighborhoods with little or no fanfare. Essentially the same service with either a domestic or commercial marketing slant, Fibre To The Home and Fibre To The Curb are names given to utilities that bring optical fibre directly to a home or building rather than delivering converged data services through a lower bandwidth copper connection.

In the old school scenario that FTTC replaces, individual subscriber’s copper, usually UTP or unshielded twisted pair, is eventually aggregated into existing fiber optic trunks serving an entire neighborhood. The more copper involved, the lower the potential bandwidth that any subscriber can expect. Cable modems or DSL (Digital Subscriber Line) services “piggyback” data on those existing copper connections we all use. Though there are small scale exceptions, this is the typical approach taken in the US of A. Elsewhere in the universe, FTTH services are usually subsidized by less myopic governments in an effort to bootstrap their national IT (Information Technology) infrastructure and, as a result, end user costs are kept quite low. This is what was done to create our original phone system but the move away from government support of public programs in the 1980’s has resulted in unchained former monopolies and short sighted profiteers, aka “Market Forces,” dictating our current public IT utilities build–out. Nowadays, for what I pay for very low rate ADSL in San Francisco, my buds in Tokyo receive several tiers higher speed service but, I digress.

Suffice it to say that, now that broadband data services are available to most locations in the US, this allows audio geeks to pass sound files over public networks without too much of a transit time penalty. Still, it would be nice to speed things up a bit and that’s where lossless codecs come in. Through the miracle of mathematics, it’s possible to reduce the size of a file by about 40 to 60% without discarding any information at all. The ways this can be done range from simple (Huffman Coding) to sublime (MLP) and, as a result, there are far too many codecs to choose from. Indeed, unless you have the marketing muscle of a company like Dolby Labs, your product can get lost in the shuffle. Claude Cellier, President of Merging Technologies and maker of the LRC lossless codec, opines, “While being probably the audio company that ‘invented’ Lossless Audio Compression in its modern form, filing for a seminal US patent as (far) back as April 1995, we are by far not even a tenth as active at marketing and promoting it…If building, enhancing, fine-tuning and improving Pyramix wasn’t drawing on most of our resources, time and energy, we’d surely be more “active” promoting LRC but days (only) have 24 hours.”

Amongst the scrum of competing offerings, a few have been standardized or mandated, such as the lossless codecs included in Windows Media 9 and Philips’ Direct Stream Transfer process built into SACD. (At press time, Apple included a lossless codec in QuickTime 6.5.1 and iTunes 4.5…Now you can store even more mixes on your iPod without comprimising quality. - OM) The aforementioned Dolby controls MLP, the lossless codec mandated for use in the DVD-Audio format and their sometimes strident promotion of the format keeps licensing fees flowing in. But there are many other codecs out there, including several Open Source choices. FLAC or Free Lossless Audio Codec competes with the less developed Monkey’s Audio and WavPack for developer’s attention. There are also closed or proprietary applications, such as La, LPAC, Shorten, and OptiFROG. Shorten, in particular, is quite popular with hobbyist music traders. In keeping with their vision of a universal media standard, the MPEG machine is also adding a lossless option to their body of standards, Audio Lossless Coding or MPEG-4 ALS. According to Tilman Liebchen, one of the authors of the LPAC codec,“…an improved version of the LPAC algorithm was recently chosen as (a) reference model” for MPEG-4 ALS.

Modern lossless codecs are rather complicated affairs but, basically, they often work by applying carefully selected filters to the audio, noting the filter coefficients and only storing the coefficients and residual audio output of those filters. There are, however, some simple forms of lossless compression that you probably use every day to streamline your work and you may not even know it…One is the Zip file format, created by PKWARE and used everywhere to reduce file sizes for transmission over the ’net. Another is run length coding, which is built into data tape drives, their so called “hardware compression” option. Run length coding consists of the process of searching for repeated runs of a single symbol in a data stream or file, and replacing that run with a single instance of that symbol and a run count. Here’s an example; at the end of a song recorded at 44.1 kHz, let’s assume that there’s two seconds of digital silence. So, instead of explicitly writing 88,200 identical samples to tape, a backup would write the equivalent of “zero amplitude sample 8.82EE4 times” instead. You can imagine that this would save a fair amount of space on the medium.

Last month, I mentioned some products that I’ve been listening to and I wanted to discuss one…The folks at Shure have a new series of consumer, in–ear monitors that basically rebrands the transducer component of their professional personal monitoring systems — same products, different package. At last year’s Home Entertainment show, I briefly listened to the E3c and E5c and “understood” the timbre immediately. Their marketing crew subsequently gave me a pair of E3cs to evaluate and I have to say, if you want to replace your iPod ear buds with something much better, buy an E3c. After a short break–in, I was surprised by their low distortion, lack of resonance and unhyped character. They’re comfortable enough for extended wear and, since they provide very good isolation, it allows you to monitor at lower volumes. Also, if you want something with even lower distortion and extended frequency response, audition the E5c.

That’s all for this month’s technobabble. Next month, I’ll head back to the pedant writ large, my Pedants In A Big Box IT glossary. In the meantime, grab your pocket protectors and continue to rock!

Bio

This month’s column, my 50th installment of the Bitstream (Oy!), was written while multitasking at O’Reilly’s ETech 2004 conference in southerly San Diego, where I filled other geeks in on UWB basics. During the controlled chaos known as a Southwest flight, I ate the welcomed packets of peanuts and soaked up the sounds of Charlie Hunter and his friends.

Pedant In A Box

This month’s timely technobabble includes…

Huffman Coding

Huffman Coding, a member of the entropy coding family of algorithms, employs a simple concept with a wide range of applications in data storage and transmission. Entropy encoding uses statistical symbol substitution to quickly and effectively reduce file size. This is equivalent to taking dictation with shorthand rather than writing out each and every word. Here’s how it works: first, you build a probability table for all the “symbols” in a particular lexicon. These symbols could be the letters of the alphabet or, in our case, the 224 or 16,777,216 amplitude values of 24 bit AES/EBU audio. So, for each available sample value in the millions of choices, you have to decide what is the probability that any specific value will appear in a particular sound file. If you scan the entire file, you can assign “custom” probabilities for that file. Once you have those assignments, you then take the lowest probability pair of two adjacent samples, that least likely combination, and you substitute that pair with a unique symbol. This replaces two samples with one placeholder. Then, you do it again with the next least likely combination, taking rare pairs and replacing then with a smaller, shorthand equivalent.

After repeating the substitution cycle many times. you end up with a table of replacement symbols and much of the entropy or randomness removed. When it comes time to reconstitute or decode the compressed data, you simply look up each entry in the substitution table, in reverse order from end to beginning, until every symbol has been replaced with the original sample pair. Huffman Coding is often used to further compress the data that results from the filter bank processing I mentioned elsewhere.David Huffman wrote his original paper on optimizing Shannon–Fano coding in 1952. Claude Shannon, one of the authors of the earlier work, was the same fellow that provided seminal insight into the information theory that forms the basis for much of digital audio’s current implementations.

MLP

MLP or Meridian Lossless Packing is the mandated method for lossless storage of linear PCM data in the DVD-Audio standard. MLP employs several individual tools to the original data, each one providing additional compaction to the data set. Though an explanation requires most of an article in itself, suffice it to say that MLP reduces a data payload by at least 4 bits and often more than twice that. MLP has modest computational needs, which is good from a CE manufacturer’s perspective and is also able to deal with CBR, VBR (constant or variable bit rate) and mixed sample rates in a multichannel data stream. For a more in–depth explanation, check the Papers section in Seneschal’s Info Annex.