I switch between them without any thought, ride the fixed 90% of the time.
The biggest difference is not the freewheel, but the frame geometry, so one bike (the CX) feels more twitchy, but I get over that after a few miles.
However I have found I do now have a habit of going much slower around corners (to avoid pedal strike), than I used to, which doesn't flow as well when riding with a group.
Al