Big people go downhill faster than little people for the same reason they go uphill slower.
Big people might go faster on the flats because bigger people tend to be stronger than little people.
OP, try gearing up more. Practice pushing bigger gears at a slightly lower cadence and you will start going faster on the flats as you get stronger. If someone is beating you on the flats, then it generally means that they are stronger than you are. That you can climb hills faster just means that you are lighter then they are.
__________________
Cat 2 Track, Cat 3 Road.
"If you’re new enough [to racing] that you would ask such question, then i would hazard a guess that if you just made up a workout that sounded hard to do, and did it, you’d probably get faster." --
the tiniest sprinter