Our front rack is permanently mounted on the steel fork of our Speedster, but we also have a carbon fork that we use when not touring. It's not much more difficult to switch the fork (both have crown races installed and steerers cut to the same length) than it is to remove / replace the front rack.

Our rear rack stays on all the time except when using the bike to participate in a cyclosportive / gran fondo ride (which is about once a year on the tandem, we offer prefer to use our singles for those events).