Robert
Tapehead
My point entirely, as you say, "you simply compare the two" - but on what criteria? Some distortions measure well but sound awful, and vice versa. I completely agree about differing masters etc, and that is not the point.
You have to choose the criteria to fit the question being asked.
If the quested is concerned with creating the most accurate representation of the master tape, you compare the two under controlled conditions, preferably with several pairs of ears and see what if any differences are detected. A situation where the listeners cannot tell the master from the same thing on the chosen carrier means perfect accuracy has been achieved.
You could also do null tests to establish how closely the two cancel if you wanted to attempt expressing these things in numbers.
If the question is concerned with likeability, feet tapping and enjoyment then all notions of strict accuracy go out of the window.
Now there are some who say that accuracy (as in identical to the source) is the absolute goal and all systems should aim for this standard. To which I would say, bollox!
If something is accurate but leaves you emotionally unmoved, while a different rendering gives you pleasure, the latter wins without question.