A GPS is great until you run out of batteries, or it gets wet or dropped and ruined. A compass (or two - just incase) takes up little room, and if cared for properly they should never get ruined. A laminated map is nice too. Also, if you get too lost and you have a compass with or without a map - good for you. If you know how to use and store a compass properly, there shouldn't be much of a problem.

I carry a GPS and a compass. I don't use the GPS on the bike because I have never needed it with the compass.