Doesn't happen frequently, but lining up up handlebars with the wheel/fork and making sure shifters are lined up is supremely annoying (especially if your bike is already cabled up). It just always seems to be ever so slightly off.
Other than that, dealing with caked on brake dust after wet rides (all the time in the PNW) is the worst.