Cyclocross - Gearing question

Granted, this bike will be for commute and cross but the hills are what's making me think.

I get about 100 or 120 miles a week and will be ramping up over summer and fall for fun and cross season. As far as power goes I'm just a regular joe I suppose.
My question is regarding the gearing choices as I build my bike. I have 12-27 rear cassette right now with 53/39 in the front. All Ultegra 9sp.

I'm ok on most climbs on the road with that gearing on my 700x23 tires, no granny spinning but still doable and able to settle into a cadence.

For cross, there seem to be far more steep sections and my MTB is geared like a full on billygoat. There's a big difference in these gearing strategies.

What should I aim for a good all around gear setup? Should I get a 32 or 34 in the rear? Go for smaller than 39 in the front? I read of folks going 1x9 with a 42 in the front, so is 39 fine?

I have a short cage RD maxed at 27t and the only med cage RD I have is a crappy Sora or a nice Dura Ace.

Will I be ok with what I have for fun'n'sport 'cross use?

How about switching to a compact crank. Maybe something like a 50/36. I've used on for cross and it did fine. Ok for the road, but you lose a little on top end.

I've not tried the compact. Will it work on 130 bcd? The money involved may make it prohibitive. I have a DA crank on the way so I'm sort of stuck with the cranks I have. I'll do a little research

No. If you have that crank, then I'd either sell it or just suck it up and use it. You may get away with switching to a 14/30 cassette. Harris Cyclery sells them. They work fine with a short cage RD.

I'll check out the 14-30 then. Wonder if a 50 large ring would be good too. The chain gets short when in the 53 and lower gears.