The GoPro is an action camera specifically created for the exact purpose that you describe. Nevertheless, if you still find it to awkward they do make and even smaller version (albeit with less control) called the Hero Session 5.

You should be able to find within your price range. There are also copy-cat models from other manufacturers and earlier versions of the Session that cost even less. It can be mounted on the top or side or your helmet.