1) You can't blame looking silly on a bike. That's up to you. I'm 6' and if someone made fun of how I looked on my bike I would help them change their mind.
2) It's just a different feel, a different center of gravity. I've come to actually prefer the handling of 20" over larger wheels.
3) You aren't on a mountain bike, but so long as your bike is well-adjusted, in good shape, tires are good, you can do most trail surfaces just fine. My wife does a few miles of gravel and dirt in her daily commute on a six year old bottom of the line Dahon without any trouble.
4) Never had the problem.