My quick and straight forward experience does not seem to support that theory, I strongly suspect it is the internet speed.
I left doing this job until the last minute and was rushed, so don't recall the file size of both the North American and western Europe Garmin files, but was pleasantly surprised that it didn't take hours (but was ready to leave it and do other trip prep).
Our internet is fast, but not optical cable fast or whatever the fastest stuff is in our area--- but again, I have no recollection of real world download speeds of our internet
Garmin’s estimates for this tend to be quite long but I’ve never seen it take anywhere near as long as the estimates. But I’m not sure it’s as fast as it could be. I’m mostly suggesting Garmin might be doing something other than doing this as fast as possible.
(I believe Garmin downloads the map files to your PC and then copies the files from there to your device.)
There are regular complaints about it being too slow. It shouldn’t matter: it should be easy enough for people to find something else to do in the meantime.
==================
If this is any sort of issue for someone, they can learn to create custom maps (using one of numerous free websites) for just the region they plan their travel to. Since these regions are typically much smaller than the Garmin maps, you don’t have to remove your home map. This approach would also let you do trip preparation ahead of time and keep your device usable for local rides. And it avoids needing to swap maps back. (Newer Garmin devices have enough space for all the maps.)
Another advantage of this is that you can often get more recent updates than Garmin provides (especially useful in the unusual case of wanted to get your own update to OSM). One downside of this is that the maps look different. Another is that you have to be comfortable copying files. Newer Garmin devices also use MTP to connect to your computer, which makes things more complicated.
This approach is easier after doing it once. I’m partial to the bbbike.org site.