I regret to say that people do report hearing a difference when there is none, as well as reporting hearing no difference when there is one.
There are four possible outcomes from a test where a subject is presented with A and B and asked are they different? These are:
- difference=YES and the subject reports difference=YES (AKA true positive)
- difference=YES and the subject reports difference=NO (AKA false negative)
- difference=NO and the subject reports difference=NO (AKA true negative)
- difference=NO and the subject reports difference=YES (AKA false positive)
I think Marky-Mark is saying that false positives happen and you are saying they did not in your tests (sorry if I mis-interpreted). However false positives do happen, in general at least, and statisticians do have to deal with them (and they know how).
I could only find an online audio example of false positives from Stereophile's tests, not of cables but of "is there a difference between
a regular CD played on a regular player (A); and
a tweaked CD played on a high-end player (B)". This is at this
link.
Each subject listened to seven different pieces of music four times. The last two times were (randomly) either A-B, A-A, B-A ,or B-B and the subject was asked is there a difference? On page 2 of the article you will see results showing from 0 to 7 right answers from each subject (true positives and true negatives). The wrong answers comprised both false positives and false negatives. Table 4 combining all subjects specifically shows they both exist.