you can have two sets of wheels, and it's not difficult at all to take off/put on wheels. i suggested semi slick tires because it would be overall less expensive and more simple to have one set of tires that do whatever you need to. by your estimates, you will be riding " on pavement 50% of the time, gravel 25%, off-road 25%." you don't need "true mountain bike tires" for gravel. so you would be buying a second set of wheels for 25% of riding.

to me, and this is just my opinion, you don't need an extra set of wheels for this. ultimately it's your choice, and there are no wrong answers on this point.