I’ve been riding CX and now a gravel bikes since about 2014 after riding a Madone for a bit. I’ve kept up with roadies fine, and did so, day of after making the switch. My first long fast ride (20mph is fast for me for 40-50 miles ) wore me out pretty good on my first ride with the CX bike but that’s because I was in the front 3/4 of the time on the knobbies. I’ve never had my legs die out like that. My point is that your engine is going to limit you more than anything else. I pushed the engine and tried to be the hero our front all day. It wasn’t my bike.
I will say big knobby dirt tires are a little slower if you’re going a long way on really smooth pavement but for most typical trials and general riding they really aren’t. I am on 700x38 grevelking now and I don’t think they are slower at all. They are a good all around tires that rolls well The CX and gravel bikes are a little heavier so there is that.
A more premium light CX bike with more aggressive geometry, full road gearing, and road slicks is going to lose nothing perceivable unless your elite level - that’s my opinion.
