Not commenting on your (I’m sure unimpeachable!) personal attitudes to race, but I think this is the stumbling block. It is for me, and I suspect Tony is catching the same issue.
Whatever a dictionary might say, in the context of a white person addressing a BAME person,m in this public way, the word deport absolutely does have racist overtones for many people if not, apparently, all.
To me, deport signifies not just an individual involuntarily leaving one place but also that the place the individual is moved to is somehow more ‘their’ place. The individual is characterised by the term deport as somehow an alien. There is a kind of relation to ‘go home’, or the state forcibly sending them back from whence they came.
Add to this the spin of it being a white person saying it to a non-white person, in a country where whites are the historic majority and people of Patel’s background have for decades been the target of explicitly racist attacks involving ‘go home’ sentiments, and the tweet become necessarily, irredeemably and unforgivably racist in tone, regardless of intent.
To be clear, I’m not trying to tell you you’re wrong or racist, I’m trying to highlight how other people apparently interpret the meaning of the word deport differently.
Personally I’m firmly in the camp I’ve described, and I suspect Tony may sympathise. Hence it’s a disagreement about the meaning of a word, not about whether you personally are a racist. It’s been an eye opener for me to see that others seem to read the word in a more neutral sense. And from the other side of the fence, hopefully this explains why some of us are so exercised about the tweet in question while others are less so.