If it sounds good and measures bad, then you're measuring the wrong thing!

If it sounds good and measures bad, then you're measuring the wrong thing! If one motto could sum up this magazine's philosophy, this would be it. Too many times we have discovered components that sounded musically fabulous while offering measured performance that was, at best, merely competent. Yet recently, I'm starting to lose confidence in that old saw.
On a number of occasions, I or another of Stereophile's reviewing team has heard a product sounding flawed in ways later revealed by measurements. A closed story, you might think—but consider the NEAR-50M loudspeaker reviewed by Dick Olsher in this issue. Despite hearing many good things in the speaker's sound, Dick was bothered by a tonal-balance problem in the low treble. He was also disturbed by a lack of integration between the tweeter and midrange unit. When I measured the '50M, my response graphs (footnote 1) pretty much explained why Dick heard what he heard. Nevertheless, other reviews of this loudspeaker have been ecstatic in their praise, one even stating that it was "one of the most transparent and balanced dynamic loudspeakers available at any price" (my italics).

The question begs to be asked: Are these other reviewers hearing what Dick and I heard? The answer must be yes. Putting to one side the problem that language appears to be an inefficient medium for conveying ideas, without an assumption that experiences can be shared by diverse listeners, there is no philosophical basis to publish any review magazine.

So why are some listeners bothered by what, to them, are gross flaws, while others are so enamored of other aspects of reproduced sound that they overlook entirely what the first listeners found so disturbing? The difference must lie in the calibration of their ears.

I have written in the past that the most reliable indicator of quality in hi-fi components is the listener's holistic reaction to the sound (footnote 2). But on what bedrock is this reaction founded? "Observation requires of the observer a considerable degree of interpretation based on expectations and already-formed models and structures," I wrote earlier this year (footnote 3).

For the listener's reaction to sound to truly reflect the performance of a component, therefore, he or she must have suitably informed his or her subconscious by becoming a) familiar with the range of performance available at any price—otherwise this phrase degenerates into empty word-spinning—and b) as familiar as possible with the real sounds of real musical instruments in real acoustic spaces.

The latter is an essential part of an audiophile's education: If you don't sensitize yourself to the differences between, for examples, the tonal quality of an oboe and that of a soprano saxophone, or the difference between a Fender Stratocaster electric guitar or a Gibson Les Paul, then you won't detect the fact that a loudspeaker can confuse their sounds—something that J. Gordon Holt touches on in a 1964 essay reprinted in this month's "30 Years Ago" feature.

If audiophiles familiarize themselves with the attributes of live sound, they can then grade changes in sound quality accordingly. (I've found this to be true, paradoxically, even if they then use artificial, multi-miked, multi-tracked recordings.) If they don't do this, however, then their value judgments will be as topsy-turvy as in the case of the reviews in other magazines of some of the components we review this month.

To return to the head of this little diatribe, therefore, when you read that a component "sounds good but measures bad," you must examine what basis the writer has for determining the meaning of the word "good." To sound good, a product must at least offer competent engineering, I feel, and it may turn out that "sounding good" does not necessarily mean the same to some listeners as "neutral" or "accurate." My role as editor of this magazine is to ensure that, for Stereophile writers at least, it always does.

Footnote 1: The graph above is not the NEAR speaker. But it, too, is an example of a loudspeaker with terrible measured behavior that was proclaimed by another magazine editor as a new reference at its price.
Footnote 2: "The Puzzle of Perception," Vol.15 No.2, February 1992, p.7.
Footnote 3: See A Second Way of Knowing: The Riddle of Human Perception, by Edmund Blair Bolles, Prentice Hall Press, 1991; and Consciousness Explained, by Daniel C. Dennett, Little, Brown, & Co., 1991.