if you are that busy on the weekends skiing you probably do need to take a day off the bike during the week.

a true recovery ride for most people is around half their normal speed. if you're going faster than 10-12 mph it's probably not a recovery ride. if 10-12mph is too slow to get you to work on time, you need to drive to work.

when you have no recovery time you wear down muscle rather than build it up.