I've only listened to its only a half dozen times for my PhD Thesis to be honest (usually with the same grimace I've reserved for Pere Ubu) as I don't think its that good an album (and I'm getting to the point where I haven't the patience for stuff that merely exists for the benefit of others recommendations).The album has fairly scant specific modal references to the Velvets (which could easily have been anything else around at the time) and while I see the comments and zeitgeist have latched onto a few references to VU, the connections as they relate to the music are tenuous at best. It predates the US new wave by a considerable time and more importantly location which is why its difficult to place it in a section marked punk when it was recorded 1971-72 and more crucially when it was written and rehearsed around 1968-1969 -- that gestation period put it in the proto-punk/deconstructed rock that underpinned the new wave.
That members of the Modern Lovers also joined The Cars and Talking Heads (also not-very punk offshoots) really makes me feel confident in my thinking that the Modern Lovers were not really in the same headspace as the later Punk bands. The evidence really does not point to them being anything other than a band proximally in the region of an empty space about to be filled, but not filling it.
My justification for kicking it back to be more a reaction to the Nuggets era rather than a progenitor of the punk era is, even taking into consideration its age, it does not sound like a very forward-looking album and definitely not in the light of punk/no-wave. The plaudits it gets are beyond me as it's no harbinger of a new movement, it was just a band that existed at a halcyon era (the correct use of the word halcyon, BTW, a moment of stillness, calm) where it could be pigeonholed into anything. JR's later hit "Roadrunner" is about as "punk" as TRB singing "2-4-6-8 Mortorway" or Blondie singing Disco.
That's my thoughts on it. Feel free to argue or disregard or whatever. I've not a lot of time for anything Jonathan Richman has to say or offer (or the Cars or Pere Ubu or that whole branch of American music).