Do Audio Measurements Matter?

An Interview with Audioholics

Let’s face it, audio measurements can be complicated to perform, confusing to understand, and results can be manipulated to unsuspecting customers. So we reached out to Gene DellaSala, President of Audioholics.com, who has been testing and reviewing hi-fi audio and home theater equipment for over 20 years to get some answers.

2) What has changed regarding bench testing in the 20+ years you’ve been measuring audio equipment at Audioholics? 

AH: Our understanding of the loudspeaker measurements needed to quantify and assess sound quality has improved greatly, mostly impart thanks to the research done by the folks at Harman. An accurate set of on and off axis frequency response curves not only can identify performance problems, but also help in deciding on the type of room acoustics needed for optimal performance. 

• See more at Audioholics: Understanding Loudspeaker Review Measurements

It’s worth noting how much better testing equipment has gotten as well.  Today’s testing equipment has unbelievable resolution necessary for the best consumer electronics available. 

3) Given that no loudspeaker can achieve a perfectly flat response from 20Hz – 20kHz and no amplifier can achieve 0% distortion, are products testing closer to those ideals? Why or why not?

AH: Both amplifier and loudspeaker performance has gotten a lot better over the years. Clean power is cheap these days thanks to the advances in semiconductors and modern design techniques. Loudspeaker driver technology has matured and even more importantly, so has crossover design and our understand of what constitutes good sound.

What has changed the most is that state-of-the-art performance is attainable at very low prices. There are now DAC’s with near or even at state-of-the-art for less than $300, in some cases quite a bit less. Even the Apple headphone dongle achieves exceptional noise and distortion performance and has excellent linearity and wide bandwidth. This is true, to a point, of speakers.  The studio monitors we recently reviewed showed many with excellent measured performance at a reasonable price.  The problem with speakers is that they aren’t quite achieving state-of-the-art in all regards.  Today’s best amplifiers, like the Benchmark AHB2 and Purifi 1ET400a are achieving noise and distortion performance that is as close to 0% as we can ever expect.  This level of performance is astounding.  In the case of speakers, some of the best monitors from Dutch & Dutch and Genelec (model 8361A) are achieving as close to a perfectly flat response as one can expect, and certainly with subwoofers could be flat from 20Hz to 20kHz to within tiny fractions of a decibel.  

4) What audio measurement standards would you like the electronics industry to adopt that would better inform consumers and allow apples-to-apples comparisons?

AH: CTA-2010 (previously known as CEA-2010) is a good standard for testing subwoofers. This is something we already do and tabulate for ALL subwoofer products we measure for easy apples to apples comparisons.

• See more at Audioholics: Bassaholic Room Size Rating Protocol

While CTA-2010 is great for subwoofers, a far more critical standard is CTA-2034 (previously known as CEA-2034). Measuring a speakers full performance in terms of its Spin-o-rama data and its dynamic capabilities tell us more about how a final system will sound than anything else.  There are far greater differences amongst speakers than there are amongst subwoofers, and these measurements are more important to help us sift through the good and bad.  Sadly, a lot of speakers do not have great spin data and very few reviewers can or do measure this, making it difficult to help manufacturers produce better products.  

5) What measurement techniques can fool unsuspecting consumers? 

AH: The most obvious will be in-room speaker measurements, especially with room correction applied.  The best room corrections, such as DIRAC, invert the room response leading to a near perfect in-room response.  Done incorrectly, they can give the impression of a perfectly flat response, which is far from honest and far from what you actually hear.

• See more at Audioholics: Audio Measurements: Useful vs Bogus

Less obvious are incomplete measurements on electronics.  For example, taking signal to noise ratio measurements of a 1000 watt amplifier can give a false impression that an average or poor performing amplifier has an excellent signal to noise ratio.  An amplifier with a S/N ratio at 1000 watts of 115dB might look really good, but may only reflect 85dB at 1 watt.  What that might seem extreme, modern high powered Class D amps are not uncommon.  Another good example is measurements looking at distortion, but failing to look at linearity.  A product could be designed to produce extremely low levels of distortion, appear very linear, yet examination of its frequency response, IMD, etc. might show it to be quite poor.  A product can’t be characterized by just one number, you typically need to look a set of metrics to fully characterize and understand the sound of a product. Similar to how we can’t understand the sound of a product by just focusing on one aspect such as its bass performance, we need to look at all aspects of its frequency response, distortion, soundstage, etc.  

6) Have any manufacturers earned your trust over the years with consistently accurate or conservative measurements?

AH: There are tons of companies that earned out trust over the years such as but not limited to Anthem, Denon/Marantz, Classe, Revel, and RBH Sound. 

7) Based on Audioholics bench tests, which products have recorded the best measurements in their respective category?

AH: A Purifi Class D amplifier module (used in the NAD M28) we are currently evaluating is one of the best measuring amplifiers we have ever tested. Philharmonic Audio’s BMR Philharmonitor is one of the most linear wide-dispersion speakers that we have ever come across. The Revel C426Be center speaker has to be the highest performing center channel speakers that we have reviewed. The JTR captivator 2400 ULF has the most powerful deep bass among subwoofers that we have measured. The RBH Sound SVTRS Active speaker system is one of the most dynamic, impactful speakers we’ve ever listened to.

8) Does “best measured” always mean best performing/sounding? If not, what are the limitations (if any) as to what audio measurements can infer? 

AH: Measurements can tell you everything about the sound of a product if you have all the right measurements and understand how to read them.  A common false assertion made is that two products can measure identically and sound totally different.  This isn’t true and there are no two products that measure identically (and if they really did, they would sound the same).  It’s important to remember that no one measurement of a product tells us how it sounds, we need a variety of measurements to understand how a product can sound.  

• See more at Audioholics: Objective Loudspeaker Measurements to Predict Subjective Preferences

It is also fair to say that we don’t have every measurement we need to fully characterize a product, and so there are things we can hear, for example related to sound staging, that can’t be fully characterized by measurements right now. At the end of the day, a measurement is a measurement and human perception is human perception.  Separately, these can’t be considered synonymous.  The science that marries measurement and perception is psychoacoustics.  We know a lot about what measurements matter for our perception of good sound, but it would be unwise to suggest we know everything, or even close.  There is still much to learn and until we have learned it all, it’s fair to say there will be things we hear and can’t measure or necessarily fully understand.  Unfortunately, because of how much we don’t know, we have left the door open for faith based beliefs about sound quality which leads to non-sensical ideas like cables or isolation devices making improvements in sound when in fact they usually don’t.     

9) What do you wish consumers knew about audio measurements so they won’t keep asking you?

AH: There are a lot of misunderstandings around measurements, many of which were covered above.  We often hear people argue that measurements can’t tell you how a product sounds, that two products can measure the same and sound totally different, that our measurements are inaccurate, etc.  At the end of the day, there are a lot of ways to measure products.  What matters most is that you follow an identifiable standard and those measurements have been scientifically proven to be associated with perception, which is why psychoacoustics are so important.  Not everything is equally important.  For example, distortion can be pretty high before it really matters, and a speaker could have a large amplitude deviation, and if the Q is high enough, it likely won’t be audible.  Measurements are a critical part of the review process, as is the subjective listening tests, but we can’t do all the work for the reader.  They need to come in with an open mind and spend some time learning about the science of psychoacoustics.  This is true if you believe in measurements or not.  

• See more at Audioholics: Amplifier Measurement Standard , Loudspeaker Measurement Protocol