-----
yes, you are correct that there is a mushroom shaped "button" which acts as a barb to secure the guide in the hole
the button has a vertical slot cut through its centre such that the "ears" have a spring action as it is pressed in and it holds in place via friction
here is an image of a model from Cannondale showing the slot in the button/mushroom:

have always used the tip of a small bladed screwdriver to pop them out without difficulty and have not broken one yet (touch wood!)
-----