What are you trying to do with a GPS? Navigate? Follow a pre-determined route? Substitute for a cyclometer? Serve as a cycling log? Many of the "cycling" GPSs don't actually navigate that well- at least in the manner you might expect. They do of course have HRM, cadence etc but are a very expensive option just for that info. I personally use a cheapo Garmin Etrex ($75 used) with city navigator (navigation) software on it. This allows me to get back home if I'm lost and also serves as a cycling computer via it's dashboard. I don't much care about HRM, cadence etc since I'm too cheap to get an SRM and they are the only decent training too.

Couple of choices- others have mentioned the Garmin etrex- I use this with City Navigator and it works fine except for planning out turn by turn routes. You can do this with GPX files but I've not found an easy way to do so. Generally I just ride with it running as a cyclometer (there's a dashboard screen you can configure to show whatever you want) and upload the data track files afterward.

If it's just navigation you want, then several smartphones such as Nokias and Android also have google or native maps available. Nokia has a sports tracker (free from Nokia) and the Android has similar programs. These generally don't feature tracking or GPX output but it depends on your needs.

There's also an Open Source android application with mapping that would be a great basis for a custom tailored cycling application. http://www.andnav.org/ I use this on my HTC G1 and it's very, very good. There are no cycling specific features.

battery life for either of these choices is not really an issue. The etrex lasts (literally) all day on a set of AAs. It's easy to carry some extras too. Same with cell phones- just a few grams for an extra battery.