We basically do if you look at some of the frames that are available now days.

There are many differences for many reasons.
Weight - (road frame advantage)

Aerodynamics (road frames are improving here, but it is still a balance of weight, and need...don't really need heavy aerodynamicly efficient features while climbing at low speed...you need low weight)

Regulations/rules (UCI got snippy about 15 years ago and started telling everyone what they thought a road bike should look like, and what TT bike should look like)

Geometry - you need different seat tub and head tube angles to allow for the bikes to be ridden comfortably and still steer safely depending on what types of bars being used. Remember that aerobars are illegal for mass start events...because they're dangerous in mass start events.

I like the dragster comment. Ever wonder why different motorsports vehicles look and function differently? It's because they are purpose built. Just like road vs TT frames.