Sighted vs. Blind Evaluations of Audio Gear, part 2

With speakers, trained ears that know what to listen for are the ultimate measurement tool. However, with electronics, our preconceptions of what we think, want, or expect to hear often obscure measurements – and reality.

I learned that lesson dramatically when I was a student at MIT in the ‘60s. I was taking a psychology course. As a term project, we were required to design and carry out a test to statistically accurate results of less than a 1% error rate – about the same chance as flipping a coin and getting heads seven times in a row, possible but very unlikely. Being into audio – I had started Tech Hifi by then – I chose to evaluate the perceived sound quality differences between two amplifiers with similar power specifications: one a very expensive – for students – high-end model and the other a popular $120 DIY kit.

The listeners were students interested in audio. I also got other students to lead the tests, so I would not influence the listeners. They listened through a pair of AR3a’s with a Thorens turntable, top-of-the-line Shure cartridge and high-end preamp. The levels were carefully matched to within .1dB. The set-up allowed the listeners to switch from one amplifier to the other instantly with a light indicating which amplifier was playing. The listeners were allowed to take as much time as needed to determine what if any differences they could hear.

They chose their own sources. The test sessions were with one to four listeners at a time and groups were allowed to discuss their impressions before handing in their reports. I had explained to the leaders that I was trying to determine if peer pressure could affect opinions.

The results were enlightening. Every listener in every session could hear improvement with the expensive amplifier and usually used similar terms to describe it like “tighter bass with better control.” “extended and opener treble” and “more detail” on voices. There seemed to be no peer pressure as nearly all reported distinct improvements of similar nature. Only one listener found the sound to be “very close;” but noted the same type improvements with the more expensive amplifier.

Although the listeners and the testers both thought it was a sighted test, it was actually a double-blind test of preconceived ideas as they were always listening to the same amplifier since all the switch did was change the light. I had enough listeners that even calling the single “close” as a “no,” I was well below the 1% error factor. My conclusion: preconceived opinions are very powerful.

Enjoy!

John

John Strohbeen Author

John Strohbeen was the President and Chief Engineer of Ohm Acoustics from 1978-2023.