If by 'records' you mean vinyl then Truck Store on Cowley Road (10 min walk from the centre of town) is the only real option in Oxford for new vinyl. They also sell some secondhand vinyl plus CDs and DVDs.
HMV still exists in the centre of town, small CD section in the basement plus a handful of vinyl albums. Focuses mostly on DVDs and computer games.
Oxfam, in Broad Street, in the centre of town, sells secondhand vinyl in their basement as does the Oxfam in Summertown (upstairs). As elsewhere around the country though the price is relatively high for the condition of the stock.
For classical CDs, Blackwells' Music Shop in Broad Street has a good selection.
Thats it apart from charity shops.
If you are passing Wantage on you way to or from Oxford there is a secondhand record shop (vinyl) there which has a small, interesting selection of new stuff.
regards
Kevin