advertisement


Anyone tried blind testing DACs?

Tim,

Just so it’s clear, I’m not arguing in favour of audio foo or expensive kit, add-ons and tweaks that do little or nothing. My point is simply that if I merely think I hear an improvement, when in fact there isn’t one, why do many objective fishies get worked up about that?

My perception is *my* reality.

I have a family member whose suffering from ADD, depression and anxiety and many of the real treatments haven’t helped much, so I’m more than willing to see if placebos do. If there’s an improvement without pharmacological intervention it’s still an improvement.

Joe
 
Let's be clear here. The point is not about blind testing, it is about the particular experiment design for the form of testing most regularly proposed. This is one that has the organisational advantage of generating statistically significant data from a low number of participants, but with an unstated axiom that everything apparent about the performance of audio equipment is immediately apparent in that context.

As a reductio ad absurdam, if I was sitting in the middle of a Mariachi band wearing a Carmen Miranda hat and clogs while having toddlers pulling my hair I might have difficulty differentiating standard age Oban from Cragganmore, although Laphroig might still be obvious. Experiment design matters.
 
Just so it’s clear, I’m not arguing in favour of audio foo or expensive kit, add-ons and tweaks that do little or nothing. My point is simply that if I merely think I hear an improvement, when in fact there isn’t one, why do many objective fishies get worked up about that?

I am not bothered for you as an individual; I am bothered by the impact on the industry, and by implication, on all audio enthusiasts.

Sorry if that was not clear.

Tim
 
Double blind testing only serves to prove that you can reliably detect a difference between two or more devices under test.

It doesn't tell you which sounds better, thats something you decide only after you prove you can hear a difference. If you cant statistically hear a difference then any decision you make isnt based on how they sound.

If I can't hear a difference between two bits of kit then I'll just choose the one that I like the, look, price features and function of the most.

If I can hear a difference between them, guess what, I'll choose the one I think sounds best, assuming its looks, price, features and function tick the boxes.

Point in case, I could clearly tell the difference between my iancanada dac and the sgd1 and smsl400, the last two sounded the same, and better than my current dac, eventually I'll buy the smsl because the look, features and function justify the price.

Some people are willing to buy based on sound alone, I suspect a tiny majority. Most of us have other drivers.
 
Tim,

I am not bothered for you as an individual; I am bothered by the impact on the industry, and by implication, on all audio enthusiasts.Tim

Audio is such a mature field that if you want to hear a real improvement, you need to get some vintage kit.

Joe
 
Just to come back to this point a little more: the scientific method requires the designing of an experiment or a test, to prove or disprove a hypothesis. As part of that design, the tester is required to think about what confounding effects may be in play, and design them out, or control for them. If I were designing a blind test, I’d want to think about whether there might be any of the factors I mention, and control for them or otherwise show that they weren’t valid. If I didn’t do that basic due diligence, my experiments would lack sufficient rigour and the results might not be useful.

It’s the same here. Advocates for blind testing seem not to want to verify that it’s a suitable tool for the job, but instead just to assert that it is. This doesn’t help their credibility as advocates for a scientific approach.

Try it. Define a scientific hypothesis (i.e. one that can be measured) that predicts a particular quantity that can be measured in an experiment or set of them. If you make the effort you will find you have not put together a "blind test". The experiment designed to measure a particular quantity may involve a form of blinding (different ones depending on what is being measured) as part of the experimental procedure but experiments are normally referred to by the hypothesis and what is being measured and not part of the procedure. There is no blinding sensitivity or such like because the form of sound perception is defined by the blinding used. It is exact. OK this requires one to accept the mainstream scientific view of sound perception rather than some magical audiophile invention but this is going to be unavoidable if you go through the process of trying to put together a hypothesis and an experiment. It is not trivial but the process of getting your ducks in a row will force you to be precise about quantities that can be measured rather than vaguely associating quantities with other quantities in the way required for audiophile nonsense to work.
 
I am aware, from my own personal experience, that I listen differently when testing blind, than when auditioning sighted. I can’t help it. I listen for differences under blind conditions, not simply to gauge my enjoyment. One is natural, the other is a bit forced and analytical. Why would I not try to control for that?
 
Robert, do you have a price bracket you work within? Thinking about build quality, stuff like that.

I'd say £100 and up - but not too far up!
I wouldn't spend more than £500, and for that I'd want additional functionality and a nice build/case. But I wouldn't expect to be hearing new details, rediscovering my music collection or rediscovering my inner self for that extra £400 ;)

The dac I use mostly is an Audioquest Dragonfly *Black which I just connect to my phone or tablet via USB C.
Usually get those on sale for about £80.

I'd avoid the cheap as chips unknown brand stuff - because without testing you really could end up with sub-standard performance.


* The Black outputs at 1.2v instead of the usual 2v - and since I run vintage 70s amplifiers and receivers which have more sensitivity on the inputs than modern kit, that works very well.
 
  • Like
Reactions: MVV
No need for a blind test between the Project dac and SMSL M400 they both have different presentations.

I have to confess, I have no idea what "presentation" means in terms of audio jargon. At least, I hear it used in the sense "it sounds different but not necessarily better or worse." On the other hand this jars with my sense that what we are after is accuracy and that both are unlikely to be equally accurate. I appreciate that when it comes to something inevitably not very accurate, like a loudspeaker in a living room, maybe "presentation" is as good a word as any!

Tim
 
Last edited:
Tim,

Just so it’s clear, I’m not arguing in favour of audio foo or expensive kit, add-ons and tweaks that do little or nothing. My point is simply that if I merely think I hear an improvement, when in fact there isn’t one, why do many objective fishies get worked up about that?

My perception is *my* reality.

I have a family member whose suffering from ADD, depression and anxiety and many of the real treatments haven’t helped much, so I’m more than willing to see if placebos do. If there’s an improvement without pharmacological intervention it’s still an improvement.

Joe
Yep - hence Placebo *Effect* (not the Placebo Illusion or whatever).
 
Hi Tim, when I use the word presentation it implies everything is there it's just the spatial alignment of parts of the the track are different. Imagine going to the theatre two nights in a row, and the second night everyone stands in a slightly different place. ie. Some peices of hifi bring the vocalist forward and others drop them back, it's not just a source contribution. I have no idea which one is accurate in that respect, we just decide which we like more. I see accuracy more in line with how things sound, "does that drum sound like a drum?"

Gavin
 
Thank you all for helping me learn how to skim threads :0) I feel confident in my skimming abilities now, seconds it has taken me to reach page 7 and I'm sure I've missed nothing important at all.
 
Try it. Define a scientific hypothesis (i.e. one that can be measured) that predicts a particular quantity that can be measured in an experiment or set of them. If you make the effort you will find you have not put together a "blind test". The experiment designed to measure a particular quantity may involve a form of blinding (different ones depending on what is being measured) as part of the experimental procedure but experiments are normally referred to by the hypothesis and what is being measured and not part of the procedure. There is no blinding sensitivity or such like because the form of sound perception is defined by the blinding used. It is exact. OK this requires one to accept the mainstream scientific view of sound perception rather than some magical audiophile invention but this is going to be unavoidable if you go through the process of trying to put together a hypothesis and an experiment. It is not trivial but the process of getting your ducks in a row will force you to be precise about quantities that can be measured rather than vaguely associating quantities with other quantities in the way required for audiophile nonsense to work.
A scientifically useful, reasonably sensitive test is possible. But that's a major undertaking and we punters have no chance.

In any practical sense for home audio, blind testing is a valuable part of the picture but not the whole picture.

Sighted listening is flawed too, we all know why. But still useful.
 
Last edited:
You can not like it; but blind testing is our best hope in the quest for the most accurate sound reproduction at home. Otherwise we are the mercy of the foo merchants.

Tim
Bollocks, not an inveterate box swapper and have had not that many systems over the last 50 years. If you need blindfolded to hear it, you've probably been hoodwinked by Foo merchants and wouldn't know good foo from bad foo..................
 


advertisement


Back
Top