'base building' is just another way of saying/getting thru the 'offseason' with as little falloff as possible.
Going totally into alternate training activities would cause a huge dropoff in technique and cycling specific conditioning. I;m not kidding myself into thinking that if I do some skiing, it will greatly improve my cycling - at best, maybe I don't have as large a dropoff as if I went bowling instead.
You have to do something cycling because if you take a week away from the bike, you will rust and decay.
SO maybe you throw down some steady miles at a level which works you aerobically, but doesn't cause huge stress from high effort. SO Maybe you do a few shorter intervals to keep some shortterm power and snap.
'base building' is a really good idea for a rider doin 30 hours + inseason, and still doin 30 hours in off-season base building.
I don;t think any of these 'base building' gurus would say a 12 hr/wk inseason rider is gonna build a strong base while riding 12hrs/wk of 'base building' miles.
So what are gonna do? Of course, you're gonna quit your day job, do 30+ hr wks base building and become a pro...
If you're a Cat 5 or 4 wanting to step up the game and you're limited on the time you can ride, then the off-season can be a time you can work on aspects of your toolkit. And coming into the new season a bit stronger than the year before, without feeling mentally cooked, might be what is needed to be a better racer, step up a cat and maybe win some races.