You say you are interested in low cost and small size, but not sure what your actual budget is. I second the Pentax Optio. They rock. Mine goes everywhere with me, and fits in an Altoids tin. I pop the Altoids tin into a small Crumpler John Thursday bag and velcro it to the handlebars.

I have the S5N (5 MP) but there is a newer model out now, the S5Z. It's $219 from Amazon, for example. Only downside is these aren't waterproof like Bek's model.