Sighted vs. Blind Evaluations of Audio Gear – part 1

Probably for as long as there has been a world of audio, there has been the ongoing debate between those who believe in “sighted” testing and those who believe that “blind” testing is the only way to truly evaluate gear.

“Sighted” testing is when the person leading the test and those doing the evaluation are aware of precisely what is being tested. For instance, you’re trying to determine which bakery produces a better chocolate cake. You have labeled samples from two different bakeries and ask those doing the evaluation which they like better.

There are two different types of “blind” testing. In a single blind, the evaluators are unaware of what is being tested. A double blind has both the evaluators and those leading the test in the dark.

Let me state at the outset of this discussion that I am a proponent of both sighted and blind testing. I feel that blind or double-blind testing – when possible – of electronics, cabling, crossovers, etc. is the way to go. But in terms of speaker evaluation, it is nearly impossible to do double blind testing of speakers – particularly where it is most important: in the home. So, we must rely on sighted evaluations. That said, I believe that a pair of well-trained ears and a quick mind are the best audio processors currently available – far better than all the test gear combined.

A trained listener can almost always tell that there is a difference in two speakers if they don’t have the same shape and components. Usually, they even hear a difference when two identical speakers sit next to each other because the room/speaker interaction is so great. Which is better is a more difficult question. A trained listener, armed with knowledge of the sources, can point out errors that most would miss.

A case in point was Dick Olsher’s Stereophile review of the Ohm Walsh 5 Limited Edition. Dick used recordings of his wife’s singing as part of auditioning the speakers. These were recordings Dick had made himself. He noted that his wife was not as prominent – forward – as she had been during the recording sessions. Her voice was in the same range in which we at Ohm had measured a peak in the speaker’s frequency response (about two octaves wide and plus 1.5 dB at the center). When designing the speaker, we built a filter to smooth out the response, did blind testing and could not hear a difference. So, we kept the filter because it measured better and did not affect the sound quality – we thought. Dick heard that we were wrong. We put a switch on the filter, did not tell Dick which position was the original or modified and he still said the voice was more accurate without the filter. So, we changed the design, ongoing production and retrofitted the change to anyone that wanted. Dick did a follow up review noting the more accurate reproduction of his wife’s singing.

Now, that was a clear case of a sighted evaluation, confirmed by a double-blind test by a trained listener, picking up something that we had neither heard nor anticipated. And, as I said, with speakers, trained ears that know what to listen for are the ultimate measurement tool. However, with electronics, our preconceptions of what we think, want, or expect to hear often obscure measurements – and reality.

More in Part 2.

Enjoy!

John

John Strohbeen Author

John Strohbeen was the President and Chief Engineer of Ohm Acoustics from 1978-2023.