I do a lot of bike path riding and I regularly use the iPod shuffle. It's either the first or 2nd generation... holds 240 songs, it's the size of a book of matches. The newer versions hold a lot more songs now...
I just put velcro on the back of it and attach it to a little plastic holder rigged out of a reflector mounting bracket on my handlebars. No video to distract you with either. And the Shuffle has a neat little built-in clip so you can clip it on your clothes (chest pocket, open jacket, etc).
I also use an old Sony am/fm Walkman. I usually switch to the iPod when the radio stations all start playing commercials at the same time (it's a conspiracy I tell you!).
I prefer cheap over the ears headphones. Ear buds seem to suffer from more wind noise than the over the ear style headphones, IMO.