All the top clubs in the Chicago area frequently have riders on cross bikes during group rides. If there was a perceived issue with riding a cross bike I would blame the perceiver.

Many factors control bike feel. Could just be that your cross bike nailed all of the things that make you feel good on a bike. Slap some road tires on it and see if it continues.