I used to own a cheapy helmet and never wore it. While it was the right size for my head, the fit just plain sucked and it was never very comfortable. A baseball cap is what I wound up wearing since it was form fitting, lightweight, and kept the sweat out of my eyes. Went into my LBS a few months back and tried on a few higher end model helmets. I was shocked at how nice they felt...bought a Bell Sweep R and wear it 95% of the time.
For me, the fit and comfort made the extra $$$ worthwhile.