Start with the top end. 11t is the smallest rear cog, so with a 46t front you're at 110.3 GI which will get to about 33 MPH @ 100 RPM cadence. If that's not fast enough for the road, you'll need a bigger chainring. Since you say you spend most of your CX riding in the small ring, you could swap the 46t for a 50t (without changing your whole crankset) and add about 9% to your top end without having much impact on your CX riding. I'd probably ride with the 46t for a while and see how it works out.
As far as the road cassette goes, you say you'd like a little lower gearing for riding to the trails, so I'd suggest getting a lower range cassette for your CX wheels (maybe 11-28) and moving the 11-25 to your road wheels. That would give you 38 GI on the low end for the road setup (which should be fine unless you have some really steep climbs) and 33.9 GI on the CX (about a 12% reduction) and it will only change your lowest two gears.