I was there! My first ever "prom". I hitch-hiked down from Bradford and back up the same night. Arrived just in time for 9am lecture, still buzzing. An amazing experience.
The 1st is a good place to start, lots of great versions available including the Solti mentioned above. The Kubelik is a personal preference, he emphasises the bucolic side and is less of a show-off than some.
My all-time favourite is the 9th, one of the greatest of all symphonies and a desert island disc. I've written before here about the proms 1992 (?) performance with Abbado and the BSO. He has recorded a DVD with the Gustav Mahler Youth Orchestra [think Berlin Symphony Orchestra under 23 team]. The Karajan live version is also superb.
I also particularly enjoy the 2nd and 6th but mainly as live experiences. The songs are well worth exploring too, and help make sense of the symphonies (many of the themes are recycled).
I'm not a fan of the 8th, despite seeing Tenndstedt and the LPO do it in their prime, it never clicked with me.