Quit worrying about ridiculous things on internet forums.

A LOT of spacers (we're talking giraffe-like here) indicates either a poorly fit frame, or a rider with some flexibility issues.

But there's nothing wrong with using spacers to fit your handlebar height to your body relative to the frame. Back in the days of quill stems (before spacers) you'd just raise or lower that piece. Now that threadless steerers are de rigeur, this "stem-slamming" nonsense is starting to spread around.

So yes, it's a ridiculous question. Go ride your bike, spacers and all, and enjoy it because it fits you. Or; don't remove spacers because it might look better on an internet forum, remove spacers because it will solve a fit issue.