Art Dudley on blind 'tests'

[Joe Hutch]

Originally Posted by Joe Hutch View Post
My degree is in English, quite simply the best subject there is.

now that's a subjective statement

No, it's a simple statement of fact. All the best people study English, plus you get to sleep 90% of the time the hapless medics and mathematicians are studying.

 

[Joe Hutch]

I never thought of the idea of doing a degree to get a job; thats almost as bad as working!

The only reason I did a degree was to get out of working for a living. Looking back, I should have spun out the studying longer than I did.

 

[sbgk]

Originally Posted by Joe Hutch View Post
My degree is in English, quite simply the best subject there is.



No, it's a simple statement of fact. All the best people study English, plus you get to sleep 90% of the time the hapless medics and mathematicians are studying.

it was a play on words, hoped an English graduate would see that. 1-0 to the engineers.

Don't think I've met an English graduate, any reason you're at pains to emphasise how good a subject it is and how good the people who study it are.

 

[Joe Hutch]

it was a play on words, hoped an English graduate would see that. 1-0 to the engineers.

Don't think I've met an English graduate, any reason you're at pains to emphasise how good a subject it is and how good the people who study it are.

No, simply joie de vivre arising from my return to studying the subject after a 40 year break. I'd forgotten how good it was.

English graduates abound in my family. There are three of us in the house at the moment, and a fourth was around earlier today.

 

[zarniwoop]

My favourite aunt was an English graduate. She had Tolkien for lectures and CS Lewis as a tutor - major bragging rights.

 

[darrenyeats]

My favourite aunt was an English graduate. She had Tolkien for lectures and CS Lewis as a tutor - major bragging rights.

"Props" ...!

 

[zarniwoop]

"Props" ...!

Quite. I omitted to mention her work at Bletchley Park during WW2 and her being the first woman to be selected for the British Aerobatics team.

 

[mattgbell]

My favourite aunt was an English graduate. She had Tolkien for lectures and CS Lewis as a tutor - major bragging rights.

You want uni bragging?

My Ancient Greek tutor was taught by Fraenkel who was taught by Wilamowitz who was taught by Wolff who was a friend of Goethe.

Oh and I shared tutorials with the current mayor of London. Which was actually shit as he never did any work. A bit of a shock for a comprehensive school boy.

 

[zarniwoop]

You want uni bragging?

My Ancient Greek tutor was taught by Fraenkel who was taught by Wilamowitz who was taught by Wolff who was a friend of Goethe.
.

Yeah yeah, very impressive. I once made Patrick Moore a cup of coffee.

 

[mattgbell]

Yeah yeah, very impressive. I once made Patrick Moore a cup of coffee.

Well, no-one can match that. No point in trying.

 

[zarniwoop]

Well, no-one can match that. No point in trying.

 

[Yank]

There's an interesting quirk that means DB IV may not be blind for all participants. Many of the blind testing advocates point out that in sighted testing the participants are subject to expectation bias and that their brains will invent differences where none are their giving a false positive. Anyone taking part in this latest exercise who subscribes to the view that bits are bits and modern DACs that are competently implemented all sound the same already knows that DACs will be the variable in the exercise. In a sense for them the test is not really blind as the specifics of the DAC are irrelevant. As expectation bias works both ways it is entirely possible that they will hear no difference even when one is present because they expect to hear no difference.

I guess it's too late to put up a curtain and tell them they're comparing speakers?

 

[neil]

I guess it's too late to put up a curtain and tell them they're comparing speakers?

Ideally you'd tell them they were comparing biscuits and incidentally ask them about the sound. Unfortunately you then run into the problem of being task focused and many won't have noticed anything playing. This was hammered home to me recently with a simple video of folks playing basketball. Half in black shirts half in white iirc. You were asked to watch and count how many passes were made by those in white shirts. At the end you were asked about a guy in a bear suit dancing through the shot. What dancing bear? Rewind and sure enough it's there.

 

[Sue Pertwee-Tyr]

I agree with this when it's done as a blind listening session & the propensity to call it a "blind test" further conditions the participants to the idea that they are under scrutiny - to stick with examinations as mentioned above - it's a pass/fail multiple choice test rather than a discursive essay style test.

This factor, of second-guessing one's first thought, was revealed & discussed in the thread where ABX positive results were reported - it's a naturally arising stress factor for most people, any time an individual feels they are being tested or can be definitively exposed as being wrong in their choice - it's an ego thing. (This doesn't arise when one does a personal blind test)

Furthermore, the argument often used is that a sighted test which shows differences followed by a blind test which no longer reveals differences "proves" that blind testing is "more valid", "more revealing", "the truth" is obviously logically flawed.

What we have are sighted test which have a bias towards false positives followed by a blind test which have a bias towards false negatives. What conclusions can be drawn about contradictory results from these two flawed tests?

That is all that's being said - both tests are flawed in their own way. Yes, some can point out obvious bias possibilities in sighted "tests" but fail to recognise other biases in "blind tests"

This ^.

All I'm asking the blind test advocates to acknowledge is that there are also enough flaws in the blind test to make the results at least as questionable as those for sighted tests. Also, that some of those flaws are difficult to recognise and correct for. Holding the blind test up as some sort of gold-standard, whose findings are somehow definitive, doesn't stack up. It would therefore be nice if its fans could stop trying to use it as a stick to beat the people whose subjective experiences are reported on here.

 

[Whaleblue]

This ^.

All I'm asking the blind test advocates to acknowledge is that there are also enough flaws in the blind test to make the results at least as questionable as those for sighted tests. Also, that some of those flaws are difficult to recognise and correct for. Holding the blind test up as some sort of gold-standard, whose findings are somehow definitive, doesn't stack up. It would therefore be nice if its fans could stop trying to use it as a stick to beat the people whose subjective experiences are reported on here.

I'd certainly acknowledge that blind tests have issues, but would argue they are fewer than for the sighted alternative.

Moreover, though, I'd hold up a combination of the two as the obvious way forward for anyone who genuinely wants to get a feel for what differences exist. I'd say this is particularly true for the folk who can hear supposedly major differences who might retreat to a more constrained set of described changes if they include some level matched blind listening.

Surely most of us would prefer to read subjective posts on equipment where this was the approach taken?

 

[Paul R]

This ^.

All I'm asking the blind test advocates to acknowledge is that there are also enough flaws in the blind test to make the results at least as questionable as those for sighted tests.

But this isn't true.

Paul

 

[kevindoyle]

I think you need to define "this pursuit".

"...CD is the nightmare of a certain kind of audiophile. You put it on, it plays, and it sounds the way it should. End of story.

Audiophiles of a certain ilk do not really want a perfect system. They want a hobby. Their motto is definitely that it is better to travel than to arrive. A non-tweakable medium that just works is anti-hobbyist. There is no hobby there.

Of course, audio as a hobby in the sense of setting up a decent system as an ongoing project was always a little odd. It is as if there were hobbyists of watches that did not keep time correctly or the like.

The ironic thing is that there is still plenty to fool with - you can work on your room's acoustics forever, and play with system EQ on and on.

What is true though is that there is nothing to BUY after a certain point...Old habits die hard. People are used to the idea that audio as a hobby consists of buying things. Hard idea to get out of, wrong though it has become."

- Robert E. Greene

 

[paskinn]

"...CD is the nightmare of a certain kind of audiophile. You put it on, it plays, and it sounds the way it should. End of story.

Audiophiles of a certain ilk do not really want a perfect system. They want a hobby. Their motto is definitely that it is better to travel than to arrive. A non-tweakable medium that just works is anti-hobbyist. There is no hobby there.

Of course, audio as a hobby in the sense of setting up a decent system as an ongoing project was always a little odd. It is as if there were hobbyists of watches that did not keep time correctly or the like.

The ironic thing is that there is still plenty to fool with - you can work on your room's acoustics forever, and play with system EQ on and on.

What is true though is that there is nothing to BUY after a certain point...Old habits die hard. People are used to the idea that audio as a hobby consists of buying things. Hard idea to get out of, wrong though it has become."

- Robert E. Greene

I like 'REG', as he's unusually level-headed for a reviewer in Absolute Sound magazine. But , like the rest of us, he can be inconsistent. For many years he was a fierce critic of the CD. Then, suddenly, he climbed completely on board the bandwagon. And he's always writing about new gear, and seems to be just as much a 'hobbyist' as the rest of us.
Most utilitarian objects, such as cars and watches, can be either consumer objects to perform a job, or they can be part of a hobby. Sometimes both. I don't like cd because , rightly or wrongly, I never thought it sounded very good. In truth, I still don't.
But that's a different matter.....

 

[Steven Toy]

I'd certainly acknowledge that blind tests have issues, but would argue they are fewer than for the sighted alternative.

Moreover, though, I'd hold up a combination of the two as the obvious way forward for anyone who genuinely wants to get a feel for what differences exist. I'd say this is particularly true for the folk who can hear supposedly major differences who might retreat to a more constrained set of described changes if they include some level matched blind listening.

Surely most of us would prefer to read subjective posts on equipment where this was the approach taken?

I once read a report in Hi-fi Choice back in the days when they did blind testing as part of the reviewing process and it was said that relatively minor differences may actually be exaggerated when they are picked up in blind listening sessions. This may in itself be a form of over-compensation on the basis that if you can pick up a given change in blind tests then it must be big.

In theory this shouldn't happen because surely what we hear is what we hear, right? Especially if visual clues have been removed. This clearly does not work because it does not align with how human consciousness actually operates. Removing the relevant visual cues does not leave the subject conscious only of the auditive. He is conscious also of the fact that the visual cues have been removed; he is conscious of their absence, and the effects of this cannot be controlled as much as we would like.

A combination of both sighted and unsighted tests may yield some interesting results but I would be reluctant to jump to any facile conclusions.

 

[Ciunas Audio]

This ^.

All I'm asking the blind test advocates to acknowledge is that there are also enough flaws in the blind test to make the results at least as questionable as those for sighted tests. Also, that some of those flaws are difficult to recognise and correct for. Holding the blind test up as some sort of gold-standard, whose findings are somehow definitive, doesn't stack up. It would therefore be nice if its fans could stop trying to use it as a stick to beat the people whose subjective experiences are reported on here.

Well it seems that this won't happen judging by the immediate replies following your post!

The beatings will continue until moral improves