Don't think you'd be the laughing stock of cycledom. I shave, never had anyone comment negatively. There is one primary reason to shave and the first time you fall off your bike you'll find out.
Other minor benefits are that it's easier to put on sun screen and the wife likes it. It might make you a fraction of a mph faster too. No big deal, if you are asking, just do it...or don't.
