I imagine there are many who voted leave who are of the poor end of society & of the wealthier end but feel disaffected by the whole procedure, maybe for a UKIP led immigration rally, who knows, I can imagine such people not voting as their lives are stable but ignorant, plenty of poor people enjoy a happy existence, wealth doesn't always bring fulfillment, but those who are affected, to a severe degree, especially over the last 8 years, will vote at every opportunity for what they believe in in my experience, those who voted in protest against Cameron will have voted against the Tories since 2010 at the very least, not all poor people are the same, they are human beings with differing emotions, they are entitled to have differing opinions, poor doesn't equal, racist leave voter, you do realise this I would hope.
There is no vicious circle, just people deciding not to vote, rich or poor, as they do not feel the need, their lives are not too bad overall, or have little empathy for people in a worse state than their own, this type are not complaining so have little need to act.