GIGO - garbage in, garbage out.
If the device is sending a bad signal, then the receiver cannot magically fix it. It can try to smooth it over gracefully, but it's still wrong.
USB has abundant speed and built in error correction. It design to work robustly and dependably.
S/PDIF has barely sufficient speed and no error correction. If the sending clock is terrible or it sends errors, then there's no way to correct for that on the receiving end.
For example, my friend has a CD carrousel with coax and optical S/PDIF outputs. We tested both into a Benchmark DAC2 HGC. The optical sounded very bad compared to they coax, probably because of a poor conversion module inside the player. There's no way to fix that in the DAC.