Let's address the fact that 5 miles is enough to cause numbness. That sounds like a severe fit issue. Even on my old bike that never fit properly it took more than 40 to get numbness. Your rides are short overall, road bikes often have a bit of a mileage commitment to get comfortable. If you're new then you may need an adjustment to get comfortable but you still need to ride the bike quite a bit more to fit it right. It could actually be causing wrist pain, a combination of weak core muscles and poor fit lead to laying weight on the wrists to avoid sinking into the saddle. Fit helps here, but the biggest problem may be you can't ride an aggressive bike if you do not possess adequate fitness.

I would start by looking at reach, not bar height. Wrist pain is often due to leaning excessively on them. Bar height, IME, is more regularly felt in the lower back. See a fitter.