You are not alone. I suffer from this affliction. The older I get the more I don't enjoy speed or heights. You have described it perfectly. I can peddle at speeds on the flat with no problem. The major problem with my last ride was some very rough roads. A minor problem that I think is contributing to this is I just put on some Conti Gator skins, and I have not gotten used to them yet.