If you are using a compass, and a steel-framed bike, or have steel handlebars, it's advisable to stand away from the bike to take a bearing. In the city, things like steel poles can upset the direction the needle points, as well.
I usually carry a good-quality, plastic-based compass. I have used it rarely to navigate, but when I have, it's paid its way. I also take note of sun direction, and especially the way shadows are cast in the late afternoon to determine where I should position my tent to either take advantage of tree shade if the morning is going to be hot, or a clear field if cold.
I must admit to never using a GPS to find my way around. I did buy one, but it never worked because it was made for the northern hemisphere, and the software download for the southern hemisphere never worked properly.