I've never tried baby wipes because I've had such good luck with the reusable swim towels I already had, things like these:
http://www.swimoutlet.com/product_p/1205.htm

Most swim towels are intended to be stored damp/wet in special plastic cases. When you finish your ride, you can "refresh" the towel at the sink with cold water so that when you dry yourself with it, it's nice and cool -- better than a traditional dry towel. Once you finish drying off & changing, rinse the towel in cool water again. Since the towel is intend to be kept damp/wet, when you do launder it with your clothes, you can take it out of the washing machine and put it straight in its plastic case; no need to dry it, ever.

I've used these things (not the brand in that URL, but similar) for years for swimming, commuting, touring, and camping. They're great.