Buy a used GoPro 3 White. They run about $100 and in 720 (30) mode their batteries will get almost 3 hours. Or you can spend $400 on a new GoPro where the batteries run for 1:45 mins.
If you want something very budget get the Polaroid Cube. Its cheap. I got mine for $77 and the image quality is VERY good.