It depends, but I would not generalize to that extent. A lot depends on the bike's design considerations regarding rear wheel removal.

For instance, one of my 2 bikes is as easy as any regular quick-release bike. It has a well-designed dropout that slides forward & backward with 4 big bolts to hold it in place (horizontal motion for chain tension adjustment), and then that sliding piece has a quick-release vertical dropout (ease of wheel removal). It's chainguard is not fully-enclosed so as to not be involved in rear wheel removal.

The other bike's fully-enclosed chainguard requires that I remove several philips-head sheet metal screws so as to be able to slide the wheel back far enough to come out of the horizontal droput for service. The dropouts are regular horizontal dropouts -- no frills. Chain tensioning is as hard as any fixie, plus each adjustment requires chainguard removal, making it a huge hassle, and of poor design -- but it looks oh-so-retro.

On any future bike I will scrutinize the frame's dropout system, making sure that it has a sliding- or cam-style tension system or similar while still permitting quick-release or other skewer systems. In fact, IMO, for any good IGH bike the true heart of the bike is the quality of the dropout and chain-tensioning system-- plus a chainguard that is not part of rear wheel removal.

Beyond that, the control linkages for the IGH in my two cases are both pretty simple: one requires a thumbscrew to be turned a few times for removal, the other a single Allen setscrew. Ease of removal of these linkages is dwarfed by the above difficulties in the case of the one bike.