your '$2000' loudspeaker is better sounding than the '$6000'
your '$4000' amplifier is better sounding than the '12000'
I would say the somebody might be on to something, or might be completely off base. It's hard to tell.
It's a fallacy to assign an inverted appeal to authority to the 'innocent party', even though they don't have a chip in the game. Someone with no interest or experience in a topic is likely to be unbiased, but that doesn't mean their decision-making skills in that topic are better, just under-developed.
Under-developed tastes tend to be more strongly flavoured, the subtleties of the topic revealing themselves to a more refined palette over time. It's why photographers discovering the joys of photoshop too often go through that early stage where everything is cropped too tight, over-sharpened and over-saturated. It's why wine drinkers have to go through the Bull's Blood phase. And it's why so many home cinema first-timers seem to shout, "I've got a subwoofer and I'm damn well going to use it!"
However, there
are many incidences of the cheaper product being the better product (you have to spend a great deal of money to get an intrinsically better sound than you get from an inexpensive Rega or NAD+PSB system, for example) and more expensive components that appear to be on a forlorn quest to find the perfect partner, failing to give their all until all the boxes are ticked (Wilson Audio is a fine example of this: the Duette and Sophia are relatively easy to install and drive, and as a result sound good in all but the most inhospitable settings: the same cannot be said of the Sasha W/P and the MAXX 3. Ultimately, the bigger Wilson speakers can deliver a better overall performance, but they are considerably more demanding of room and system). So, I wouldn't discount the suggestion that the cheaper equipment did sound better, after all.