Yes, *one* could swap them back and forth; *I* however am unlikely to do so! I squeeze racing and training in around the fringes of working and parenting 2 small children. 15 minutes of bike dealing can make or break whether I ride that day.
I hear ya, I have two young kids and work full-time too. Swapping out wheels on a bike or even swapping which bike is out and ready to ride and which is in the garage is often as much as I can manage.