I think people have already posted the important bits -get one that is undamaged, fits you well, is nicely adjustable and is suitable for you (e.g. well vented in my case). I won't bother adding get one that is certified as they have to be to be sold anyway -at least in the States. Usually you have to go to a store to try one to make sure it fits properly (much like a pair of jeans or shoes syndrome I'd say). However, I will admit to the fact I've bought my helmets online -only had one helmet I just didn't like, and I bought based on comments made in online reviews.
My current favourite: a Bell Alchera -just the right shape for my head with some really effective vents. I got mine for $34 (I use one with a mirror attached for the road) and another one I got for off roading at $22 (returns from Nashbar, clearly unworn). Very comfortable with good adjustment and excellent ventilation, definitely my best helmet I've ever had.
I'm not 100% a more expensive helmet would protect you better though; I tend to believe more expensive means more vents, lighter weight and probably a closer margin to passing the safety tests. I do wonder if some $20 less vented, heavier helmet is safer than a $150 lightweight helmet looking like Swiss cheese, just no proof of that. Another thing, sometimes I think people think more vents = better ventilation, something I've found not necessarily though admittedly usually true.
My only gripe about helmets is that I wish some of the manufacturers would stop doing the trendy colour combo things and sometimes just think about safety -it's fairly hard to get a neon bright orange/yellow/green helmet. I believe you can spray paint helmets, just never got around to it and I wonder about the durability of the finish.
And just to echo one post, I completely agree that some cheap helmets sold now for around $30 are like some of the $100 helmets 5 years or so ago!