I was a motorpace specialist in the 80s. I can't recall any big muscle-y guys involved in it. Most of us (myself included) were actually kind of skinny. That game just isn't about power.
As unbelievable as Hoy was at what he did, I wonder if he would have been a decent stayer. The races generally went for 60 or 90 minutes and could cover nearly 100 kilometers, which isn't normally the kind of effort sprinters train for.