Good points above. One other area to consider is your neck. Riding places stress on nerves in our neck, it can cause the dizzyness you are experiencing. My wife is a physical therapist and has given me a few stretches (chin tucks, rotations) to do before and after rides. Along with ensuring I drink enough, my problem with dizzyness has gone away.

By all means though if this is a consistent problem - see a doctor to be sure.