Hi
@adamdea - good comments all.
That is why I picked Ben Goldacre! I thought about pointing at Kahneman on biases - equally relevant but he didn't as far as I know ever mention anything hi-fi, and part of the issue here is that we all have biases and throwaway irrationalities, including me and BC.
I completely agree that getting subjective is a problem, but so is ignoring what looks like evidence too readily. A lot was known about the failings of our hearing long before CD turned up and yet we had that 'perfect forever' stuff being 'proven' with graphs on TV. The graphs weren't wrong, but we didn't know all the things that it was important to measure. What we did know was that a surely statistically significant group of people who were not all comparing notes had remarkably similar complaints about a lot of early CDs (and were generally regarded as weirdos/ real ale enthusiasts/ still obsessed with crystal radios).
If we are not doing good science here (and I would vigorously agree that we aren't), could we do it properly and would it help?
If I had enough friends, I could have collected (say) 100 listeners for my hi-fi support experiment. I could have literally blindfolded them (in the unlikely event they'd agree). I could then have cycled through my various trials over a week and got them to dictate their findings. Even if (say) 80 out of the 100 expressed the same preference (with 'like' being so subjective and varying) or given the same descriptions (' flappier bass', 'less stereo/ can't place musicians', 'unpleasant treble on that track' or whatever), this still wouldn't change most people's minds on the topic in either direction, just as some cable-fanciers could in theory be forced to do a similar blind test, but a complete failure to be able to pick out their beloved solid gold/ silver/ fairy-dust cable wouldn't actually dissuade them.
As you say, I am not a sound evaluation device. but my ears are surely the best things to detect how much I like some music. If I find it consistently better with (say) a record clamp, even when someone is standing between me and the hi-fi so I can't know whether the clamp is on, it could be a fluke. However, it may not be random at all.
I don't know how many times I'd have to do the experiments and what 'guess right' scores would satisfy critics (would 80/100 do?). However, I am confident that most nay-sayers (who for reasons of common sense would never try such an experiment themselves) will still be 0% convinced, whatever the numbers are, well after I have lost the will to live or continue experimenting.
That has certainly been my experience with effects I could not detect but others said they could - sometimes it seems clear that something is going on but I can't hear it, and other times I think the believers are just mistaken. However, hardened believers at both ends of the range of views seem the only ones with a need to persuade.
More important is surely that your first point is right - none of this really matters. If I delusionally persuade myself that X is more enjoyable than Y, then the delusion helps me and does no obvious harm. If I invite a few friends over and some of them notice no difference and others perceive more enjoyable music, that could just be because they can read that I am pleased or they noticed new box and made assumptions, but that too doesn't matter. At least some of the people present have enjoyed the music more, and that was the point.
I have done enough blind tests over 35 years to be fairly confident that, if I notice no repeatable difference between A & B in the first 20 minutes of a not-blind test, then spending the rest of the afternoon with a blindfold won't change that. If I do think I notice a difference but with little/ no definition or consistency, then obviously I treat it as a vagary of my brain and ears and move on. And if I think i hear a difference and keep thinking it all afternoon, whether I can see the item being utilised or not, then I could still be wrong about what is going on to create that result.
For all I know, increasing the number of black items in the room is what makes my NDX2 sound better when powered by an XPS2. It is unlikely, but I have not done a proper test with suitable sample size to disprove that. Without a good reason, why would I? Similarly, just because we think we detect that 'something is happening' with a support issue, that is poor evidence that we know what that something is.
I think that this leaves me agreeing vigorously with the thrust of
@zarniwoop 's comments. One could do a really detailed examination of the factors we think important, but it would be vastly more complex than putting a vibration detector on a hi-fi stand and playing a single-volume static note on the hi-fi, or working out resonant frequencies of vast numbers of items, and we might very well be measuring the wrong things once again. Or we could hire at great expense 200 of the best ears we can find (which certainly wouldn't include mine) and spend hours doing variations of the experiments to get really statistically significant unbiased data.
I haven't done these things, and neither have those who 'know' that this is all woo and an Ikea sideboard is all you ever need, so we don't have good science - and we all know what we think about bad science.
If I wanted to work as a salesman for a company selling uber-cables for thousands, I'd really want that work done. However, the bar for me is much lower - I just need to get good enough evidence to give a reasonable degree of confidence to one listener, me. The fact that I am harder in many areas to persuade than some is potentially relevant but not proof of anything, ditto my other listeners reaching the same conclusions.
I can substantiate the comment that I consistently liked A more than B, and that others reached the same conclusions, at least to my/ our satisfaction. I absolutely cannot substantiate any claim that 100% (or even 70%) of randomly selected listeners would conclude the same. As with so many contributors here, that is why I am more likely to type 'Give it a listen and see what you think" than "you must all buy X".
