I use a Garmin Etrex 20 for cycling, kayaking, and hiking. Small, waterproof, and a set of regular alkaline AA batts gives you something like 24 hrs of running time. For biking I load GPX Track files from RWGPS through Basecamp.
You can customize the view to show four elements of your choosing. See the pic below for my setup. I typically set the screen view at 500 feet and the track is easy to follow. I have an extra SD storage chip installed, but the internal storage is plenty big to hold many tracks as the track files themselves are very small(just a collection of gps waypoints). My unit is 6-7 years old and works fine. There is(was) an Etrex 30 that has some extra features. Not sure if the Etrex series is still in their current lineup, but there will be similar units available. There should be plenty of them around used I'd think.
Like T-MSN, I have it mounted on the handle bar with a slip-in mount. There are mounts for the Etrex available for hiking(belt clip-on), cycling..etc. I have found the aftermarket cycling-bar mounts to fit/lock into the unit to be loose. I heat them up with a heat gun and bend the lock end up a little to make them lock into the unit tighter. I also use a tether around the stem just in case it gets bumped off the mount.
Some folks like to use their phones for navigation. I haven't had much luck with that route, even though the screen size is great, as the battery dies in short order when actively navigating a complicated route. I'd rather keep my phone battery in good shape in case I need the phone for a call if I, or someone else, needs emergency help..etc..