Cav
pfm Member
Whereas you are indifferent to a null result except that you do not want one?
A challenge to ItemAudio
- Thread starter Crash!
- Start date
Whereas you are indifferent to a null result except that you do not want one?
sonddek
Trade: SUPATRAC
I don't know why that view is being applauded. I've held it for many years and assumed it was common sense to objectivists and subjectivists alike. It has little to do with the question of whether someone can reliably distinguish two pieces of equipment, and if he can't then the foo is indeed foo. To me, that's what critical listening is about. Can I identify any significant difference, and if I can, does it help me enjoy the music? The hard part is listening to the same piece of music twice without experiencing it differently. It is a gross mistake always to attribute those different experiences to the kit, or some irrelevant variable. That's Beltism.
Just because experiences are so variable, that doesn't make it impossible to assess real differences in equipment. There is an objective truth about how something sounds. Accuracy to an original recording is possible. However, responses to that truth may involve taste.
This thread is not about that. It's about whether two things sound different at all, i.e. can they be reliably distinguished. If they can, we need to improve our understanding of why, because it isn't predicted by digital signal processing theory. My money is on differing responses to different bit-perfect transports being mostly in the mind. I would be interested to be proved wrong about that, but with respect, a test run by Steven and item on one DAC and a few listeners isn't going to make a summer.
Cav
pfm Member
That must be the "tutored" sighted test then?
Person A: I heard 5 tings.
Tutor: You missed the sixth ting. Listen again.
Person A: Oh, yes there are six tings...
Ciunas Audio
Trade: Ciunas Audio
Good, so can you explain it to those who are still questioning the need for a sighted listening session?
sonddek
Trade: SUPATRAC
To be honest, I don't see a great need for two reasons. First, all the judges, as far as I can tell, are experienced audiophiles, who I would expect to be quite practiced in noticing the kind of differences we're looking for. Secondly, there are supposed to be many of them, and many chances to judge, so that any real difference ought to show up in the stats, even if fewer than all judges have been guided (know what to listen for), and fewer than all judges are capable of hearing any real difference for physiological reasons.
In other words, real differences in sound should show up in the stats, albeit attenuated, even if unfamiliarity and poor hearing affect some of the judges.
I've always understood your argument, I just think it lays the test open to unnecessary criticism and a greater possibility of the test being accidentally affected by some unforeseen bias, and it shouldn't matter because any real audible effect ought to be strong enough to leave a trace in the results, as described above. In other words, if transports have a decisive sound effect, an unguided blind test ought to show it.
Steven Toy
Accuphase newbie
Ciunas Audio
Trade: Ciunas Audio
So, basically you are going back on your applauded statement "Sometimes one doesn't notice a difference in two versions of playback until it's pointed out. Forever more it's as clear as day. There are a lot of things to listen for." Because you believe the judges are some trained golden ears who will spot differences that the normal hearing wouldn't?
You are deluded if you think that the results will be sufficient to yield a set of figures that can be statistically analysed to reveal the nuances that you claim. This test will be a all or nothing success/fail type of result - anything else will be argued to be inconclusive & therefore will be deemed a null & equivalent to failure to spot any differences.
Just hope that everyone is aware that this test is already skewed towards a null (which will be considered failure) because of the number of judges & number of listening sessions that are planned.
So, Mark, you are up against a very strong bias towards this null result which of course means that the test is inconclusive but based on the knowledge/understanding/agenda/bias that has been demonstrated in the posts on this thread will be deemed a failure!!
Jkrny, you seem to know an awful lot more about the tests than the rest of us. Also, your logic is faulty. The number of judges and listening tests have nothing to do with the result.
Plus, have faith in your mate. He knows exactly what he is doing.
Plus, have faith in your mate. He knows exactly what he is doing.
The test will be as useful and defensible as I can make it. Whatever the outcome, it won't change the world: maybe one or two (maybe me, even) will revise their opinion an inch or two in another direction. I hope similar events re-run it with different equipment, conditions and listeners. All I'm trying to do is stimulate interest in, and debate about, modern digital transports.
There will always be the possibility of accusation that the equipment and room is too good/not good enough, or the listeners are too good/not good enough, or that the test was too short/long – or that blind testing itself is too good/not good enough. But it should be interesting, non?
There will always be the possibility of accusation that the equipment and room is too good/not good enough, or the listeners are too good/not good enough, or that the test was too short/long – or that blind testing itself is too good/not good enough. But it should be interesting, non?
sonddek
Trade: SUPATRAC
I'm not going back on anything. Several judges are going to compare the two transports many times. If there were no difference, or nobody noticed the difference, then you would expect the outcomes to reflect that, in a tendency towards X being called A in half of the judgements. It only takes one judge to notice a reliable difference in order to start pushing the data towards distinguishability. If two judges spot it, you're motoring. How hard is it to spot the difference? If it's so bloody marginal that none of the judges notices it during several attempts, then you probably need to do a bigger test to make sure you aren't... ...hearing things.
As I said, the problems with a sighted guided listening warm-up is that it makes the whole thing look fixed, because it is so hard to avoid the leakage of information, subliminal or not. A positive result will not be trusted.
Sure can. It was obviously going to fail here though. Far too honest, simple and capable of a Y/N answer..m
I can promise you I don't have a massive vested interest in this: I've made a statement in line with our experience, and that of many others and I'm interested to see if it can be tested. Financially, I stand to gain . . . ooh, maybe a few hundred pounds on the bottom line spread over the next few years if the test implies that two bit-perfect transports differ.
If it seems not to well, we only tested one DAC. In fact, it might encourage more people to buy a £425 T1 instead of the £999 version: why spend more if it sounds identical? Not a big deal either way.
I'm happy the original challenge kick-started some action – thanks Crash!.
But the original objective wasn't in line with the central issue being debated. It rocketed in a bit hot and oblique. The point at issue never was: is a T1 'better' than a RPi? Much too parochial. The question is: do bit-perfect transports differ?
The revision lets us take a fair stab at contributing something useful to the big debate. It couldn't be hosted by any of the digital attack dogs for obvious reasons of prejudice.
I didn't suggest Steven for this role, but he is widely known as 'agnostic' re: computer audio, and for having a revealing system. He's conspicuously not featured in the vitriolic recent threads on the subject and has a reputation for open-mindedness – too open-minded for many of you, but in this instance it makes him the most impartial host available.
As for who 'judges', I would say the more the merrier – judging from the tone and integrity of recent posts, I'm expecting some to attend simply to post 'no difference' on every test sheet without even listening . . .
Er - memories are short. You declined the original challenge and set this one up. As to prejudice - you sell the "audiophile" stuff. Pot calling the kettle black.
However, if nothing else the test should be interesting, and, assuming all the criteria that has been discussed are met, may prove useful. It probably won't add much to the debate since there isn't a lot of debate to begin with.
However, if nothing else the test should be interesting, and, assuming all the criteria that has been discussed are met, may prove useful. It probably won't add much to the debate since there isn't a lot of debate to begin with.
Ciunas Audio
Trade: Ciunas Audio
But you see, what sonddek was talking about was stats & when it come to statistics, the number of tests has everything to do with it if you are looking for any meaningful result. That's why I say that it has to be overwhelmingly decisive for this to be a non-null result.
Everybody is focussed on sighted Vs blind sessions but seeing as that phase of the discussion is just about finished after > 400 posts, it's time to move onto the next 30 pages of posts
to discuss what results, do people consider, would be an acceptable non-null result? 100%, 60%, what value? In other words if all judges spot the differences 60% of the time, does this prove that there are differences? Does it depend on the number of tests run, number of judges? If there were no differences i.e people were guessing then, given a sufficient number of tests, you would expect 50% success. Has anybody any done such a test before & has knowledge on the results/statistics about all this? See this is statistics, folks & the number of tests matter it's called "statistical significance"Next thing to consider is a test of the procedure & test itself - can the judges hear differences blind in obviously different sounding equipment? In other words, is the test sensitive enough to reveal obviously different sounding equipment? Down to what level of differences? In scientific testing, it's called a control group but here I'm suggesting that some evaluation should be made to establish if the test can actually reveal any differences, bind?
Just these couple of elements alone bias the results towards a null result as in so many other blind tests. None of them have any real validity because they dabble in pseudo-science & hence why Mark is essentially correct - the best evidence is for people to build their own systems according to the blueprint & evaluate for themselves if there is a difference. Forget about trying to convince anybody else - it's futile!