advertisement


Art Dudley on blind 'tests'

Status
Not open for further replies.
Vital said:
I'd certainly acknowledge that blind tests have issues, but would argue they are fewer than for the sighted alternative.
Click to expand...

If you hold that these types of blind tests have fewer issues then it's simple to test this. Use the controls I suggested at the next DB IV & "prove" that the blind test is picking up the differences that have been heard using ABX tests

If it does pick up these differences, I will concede that you are correct.
If it doesn't pick up these differences you have to conceded that this blind test is flawed

What's going on in these discussions is an argument over which flawed "test" is better. One side saying they are equal & the other saying one is superior. It is up to the side claiming superiority to prove this rather than just engage in rhetoric.
 
jkeny said:
If it doesn't pick up these differences you have to conceded that this blind test is flawed
Click to expand...
You could say the same for any test - including sighted! - that has a negative outcome. If the conditions are not controlled, a negative result isn't evidence that a difference is unhearable - but certainly I agree, being a blind test is not sufficient.

Of course a positive blind result is evidence unless there's been a procedural screw up.
 
jkeny said:
One side saying they are equal & the other saying one is superior. It is up to the side claiming superiority to prove this rather than just engage in rhetoric.
Click to expand...

F*ck me! Is this still doing the rounds?

Read the AES papers from Sean Olive, Floyd Toole and others. There is a case that has been proven. That's fact. What you are trying to do is to develop doubt. It's kind of dispiriting.
 
darrenyeats said:
You could say the same for any test, including sighted, that has a negative outcome. If the conditions are not controlled, a negative result isn't evidence that a difference is unhearable.
Click to expand...
No, what is being suggested here is the use of a known audible (but subtle) difference to calibrate the test/testers. If such a difference is audible sighted but not audible blind then something is wrong with the blind testing, no?

This seems simple enough to me

But certainly I agree, being a blind test is not a sufficient condition for this.
Click to expand...
Yes but let's try to take it out of the land of debate & start putting together some testing of the test to validate the claims that are being made for it

Of course a positive blind result is significant unless there's been a procedural screw up.
Click to expand...
The recent ABX positive results that I posted showed that the energy expended in searching for screw ups engages many of the same people who use blind testing as a stick. Would this energy not be be better used in proving the superiority they claim for blind listening sessions.
 
merlin said:
F*ck me! Is this still doing the rounds?

Read the AES papers from Sean Olive, Floyd Toole and others. There is a case that has been proven. That's fact. What you are trying to do is to develop doubt. It's kind of dispiriting.
Click to expand...

I'm not trying to develop doubt - I have the same doubt already in the typical blind listening sessions as I do in the typical sighted ones. None of these come anywhere near to the rigour that Toole & Floyd apply in their tests, yet you seem to be trying to deflect away from this. If anybody wants to claim their test is "more valid" "more truthful" then include the controls that will show this.
 
darrenyeats said:
Hearing it sighted wouldn't in itself prove the blind test where you don't hear it has something wrong with it.
Click to expand...

The controls I'm suggesting are ones that Vital has already produced positive results for in an ABX test, proving that his system & hearing was capable of differentiating between the test files.
I'm just interested in using these same files as a control at DB IV to see if the dynamic of blind session listening changes this result? So I suggest listening sighted & listening blind

I would hope that other inquisitive types might also be interested?
 
Well that's more like it. But his system has changed since then, in the interest of not wasting all of our time before we start, it would have to be something he's been able to do blind with exactly the config to be used in the real DBO. A safe choice would be a difference everyone agrees should be audible, like a small level change.
 
Post 666.....must be nearly Halloween.
The following statement scuttles the basic hard-line argument.

'' The results of Double Blind tests have not been shown to be applicable to more general circumstances.''

That's it. You cannot infer from such tests that people might not hear differences in other circumstances...such as sitting relaxed at home.
What the 'objectivists' actually need is something like this proposition:

''The results of double-blind tests are also true in a wide variety of other circumstances.''
 
darrenyeats said:
Well that's more like it. But his system has changed since then, in the interest of not wasting all of our time before we start, it would have to be something he's been able to do blind with exactly the config to be used in the real DBO.
Click to expand...

Right, his system has changed but hopefully for the better & is more resolving? I'm sure he could repeat the ABX tests with the same files to check this before the real DBO (using the same system). They are known to be audibly distinguishable from other ABX tests & testers

It might elevate the session from just another get-together to something that produces data of interest
 
paskinn said:
What the 'objectivists' actually need is something like this proposition:

''The results of double-blind tests are also true in a wide variety of other circumstances.''
Click to expand...

That's what is usually attempted - see the post about Toole & Floyd, AES, etc & then look at any forum blind session & laugh at the attempts of some who use hand waving to try to slyly (without stating directly) equate this to the usual blind listening sessions. This is the trap that a lot of casual readers fall for - the mention of double blind - ohh, sounds so scienicy, must be better than "ordinary" listening
 
jkeny said:
If you hold that these types of blind tests have fewer issues then it's simple to test this. Use the controls I suggested at the next DB IV & "prove" that the blind test is picking up the differences that have been heard using ABX tests

If it does pick up these differences, I will concede that you are correct.
If it doesn't pick up these differences you have to conceded that this blind test is flawed

What's going on in these discussions is an argument over which flawed "test" is better. One side saying they are equal & the other saying one is superior. It is up to the side claiming superiority to prove this rather than just engage in rhetoric.
Click to expand...
You are joking, right?
 
paskinn said:
Post 666.....must be nearly Halloween.
The following statement scuttles the basic hard-line argument.

'' The results of Double Blind tests have not been shown to be applicable to more general circumstances.''

That's it. You cannot infer from such tests that people might not hear differences in other circumstances...such as sitting relaxed at home.
What the 'objectivists' actually need is something like this proposition:

''The results of double-blind tests are also true in a wide variety of other circumstances.''
Click to expand...
No, it just says that the introduction of expectation bias, equipment design etc may cause the listener to ignore or override blind test results.

Why are people so afraid of blind tests? They're harmless and fun.
 
avole said:
No, it just says that the introduction of expectation bias, equipment design etc may cause the listener to ignore or override blind test results.
Click to expand...
Yea, I've heard that circular argument before - all DACs sound the same & any that sound different are therefore broken - back to logic school for you

Why are people so afraid of blind tests? They're harmless and fun.
Click to expand...
Why are people afraid to use controls in such blind listening sessions to test the (so called) test itself?
 
avole said:
No, it just says that the introduction of expectation bias, equipment design etc may cause the listener to ignore or override blind test results.

Why are people so afraid of blind tests? They're harmless and fun.
Click to expand...

. You need to address the actual point .Tests done in highly artificial circumstances may not read-over into more general situations. It's not exactly rocket science.
 
jkeny said:
I'm not trying to develop doubt
Click to expand...

Yes you are.

It's pointless arguing about this because you NEED people to believe Dacs make a difference to make a living and audiophiles NEED to believe that there is a sonic reason for their purchase.

There's no more rigor to the standard Harman loudspeaker test than there would be to any unsighted bake off. The test has been taken by hundreds if not thousands of audiophiles, music lovers and journalists and the results have been consistent. Or are you now saying that these tests are flawed? In which case what is it that you know that Harman International and the AES don't?
 
DBO 4 or is it 3A(ST) maybe even 3B(JK) is about enjoyment, nuts or whatever and beer, wine, etc. It is Vitals home and environment. How he wishes to conduct it within the constraint of taking other attendees views into account. Different DACs to DBO 3. It isn't intended to be a scientific peer reviewed study subjected to the strong views of the rest of PFM.

I see posts which are loosing that plot.

I am also certain some will hear an amazing difference and some may minimise the significance of the difference. It is one thing to use kit that is chosen to detect difference and how the vast majority of lesser equipment behaves with these same DACs.

Want to try a £35 Maplins USB DAC Vital - well regarded by a number of PFM members?
 
ABX in this context would be blind, so you want blind tests to justify blind tests?
 
Status
Not open for further replies.


advertisement


Back
Top