Sure it's possible, if anything these guys are selling it short. Look at prepaid phones at Best Buy for example, $20-$50 will do much of what you want. Forget cell and data if you want to, just use wifi and download a GPS navigation app, music player, bike speedometer app and you're set.

You probably won't get altimeter and a way to see cadence, or magnetic compass, with a phone like that. Just count on GPS. GPS can track elevations, just not very well IME. The app can display how fast you're going, and how far. You can do Strava with it, or RideWithGPS. You do not need cell or data for GPS navigation.

If you do want inexpensive service though there are options. I use republic wireless which is unlimited talk, text, and the slight amount of data I use, costing me $13.50/month. Or Tracfone prepaid, which you can arrange for about $7/month. Or wifi only voice and text, which if you dig into it can be had for basically nothing.