Art Dudley on blind 'tests'

[merlin]

The hifi industry thrives on doubt.

Doubt is good.

Removal of doubt is bad for business.

It is the industry's job to plant the seeds of doubt in the minds of the impressionable. It's science's job to make advances and to expose snake oil for what it is. Doubt tends not to thrive in the science world for long. That's surely not good for business?

 

[Tigerjones]

Why the hell someone would want to sit listening to different amplifiers with a blindfold over their eyes is a complete mystery to me. The best thing about amplifiers is the way they look, otherwise they'd all be in boring black boxes and not have beautiful wooden cases.

 

[Rodney gold]

If , in audio , you think you hear a difference , it is a truism for you , whether there is any difference or not
In your brain and head you hear it , and you can quantify those differences and even do a cost/benefit analysis on them.

I was once fiddling with a parametric equaliser and I was working in the low bass , sweeping center freq/gain/slope.. I got it just right with one setting. I then looked and saw I had bypassed the parametric. I HEARD the changes I expected , but they were in my head. My brain synthesized them for me!!!

What we as humans experience when listening to music is not merely via the ear , the vibes hit your body , what mood you in , what music you listening to etc, expectation bias and so forth and they are all real to the listener whether they exist in the real world or not.

 

[Julf]

Abx dbt makes Budvar, Stella and Heineken taste alike. That in itself discredits abx dbt one-and-for-all.

And in subjective tests, the same tap water out of a garden hose tastes completely different depending on price and description. That in itself discredits sighted comparisons one-and-for-all.

Penn and Teller Water Bottle Survey

 

[Joe Hutch]

Surely blind tests are the ultimate in subjective tests, because all that you have to go on is the sound reaching your ears, rather than extraneous stuff like measurements, specifications, cost, manufacturer's name, etc etc?

 

[darrenyeats]

Blind and double-blind testing have their place, but I'm starting to think that level matching is the more important variable to control when testing, whether blind or double blind.

+1.

 

[ItemAudio]

Blind and double-blind testing have their place, but I'm starting to think that level matching is the more important variable to control when testing, whether blind or double blind.

Actually, I don't really care. Buy whatever makes you happy, but if you're testing something this implies that proper controls are in place.

Joe

If you're testing a source component, the level-matching procedure needs to be much more rigorous than the half-assed attempts I've seen in most auditions. If you're testing an amplifier or speaker, forget it - you'll do as well to listen carefully and twiddle up and down, relying on that unreliable audio memory.

 

[ItemAudio]

Surely blind tests are the ultimate in subjective tests, because all that you have to go on is the sound reaching your ears, rather than extraneous stuff like measurements, specifications, cost, manufacturer's name, etc etc?

This is a point I've connected with before: blind tests are listener tests, not equipment tests.

Failure to make this crucial disctinction entirely undermines pro-DBT forums like Hydrogen. So many learned folks driving themselves to such lengths barking up the wrong tree.

 

[Alan Sircom]

And in subjective tests, the same tap water out of a garden hose tastes completely different depending on price and description. That in itself discredits sighted comparisons one-and-for-all.

Penn and Teller Water Bottle Survey

Yes, because good science requires experts in sleight-of-hand to validate its findings.

I mean, I think the whole designer water thing is nonsense, but Penn and Teller? What's next? Paul Daniels on string theory?

 

[ItemAudio]

Paul Daniels on string theory?

You're confusing Pauls: http://en.wikipedia.org/wiki/Paul_Davies
One thinks about putting cats in a box, the other Debbie McGee.

 

[avole]

This is a point I've connected with before: blind tests are listener tests, not equipment tests.

Failure to make this crucial disctinction entirely undermines pro-DBT forums like Hydrogen. So many learned folks driving themselves to such lengths barking up the wrong tree.

Albeit with better spelling skills

 

[Grumpy]

Oh dear, I think I'll put some music on instead!

 

[Joe P]

Item,

If you're testing a source component, the level-matching procedure needs to be much more rigorous than the half-assed attempts I've seen in most auditions.

I don't do anything half-assed. I use my whole ass or none of it at all.

Joe

 

[ultrawomble]

This is a point I've connected with before: blind tests are listener tests, not equipment tests.

So, for example lets take the scenario of a comparison between 2 DACS. If the listeners are being tested and 99/100 fail the test what does that tell you about the listeners?

 

[ItemAudio]

Albeit with better spelling skills

Heck, is that not how it's spelt? That's how I say it: disctinction . . . try it: it makes you sound so disctinguished!

 

[ItemAudio]

So, for example lets take the scenario of a comparison between 2 DACS. If the listeners are being tested and 99/100 fail the test what does that tell you about the listeners?

That they're able to state the obvious.

 

[Steven Toy]

And in subjective tests, the same tap water out of a garden hose tastes completely different depending on price and description. That in itself discredits sighted comparisons one-and-for-all.

Penn and Teller Water Bottle Survey

I said beer not water. Furthermore, Stella, Heineken and Budvar are all about the same price in the UK.

 

[Joe Hutch]

So, for example lets take the scenario of a comparison between 2 DACS. If the listeners are being tested and 99/100 fail the test what does that tell you about the listeners?

I'm not sure how you can 'fail' a listening test; that implies that there's a correct and an incorrect answer.

 

[paskinn]

Why the hell someone would want to sit listening to different amplifiers with a blindfold over their eyes is a complete mystery to me. The best thing about amplifiers is the way they look, otherwise they'd all be in boring black boxes and not have beautiful wooden cases.

I rather subscribe to this view. My valve amp is sprayed in a 1958 Cadillac colour 'Daphne Blue' which was also used on early 60s Stratocaster guitars. Looks great, much more fun that a black box.
It certainly makes it sound better...to me.

 

[Julf]

Suggested reading: ITU-R BS.1116, Methods for the subjective assessment of small impairments in audio systems.

Especially interesting are the sections on control conditions, selection of listening panels (with pre- and post-screening) and double-blind methodology.