There's really no rational reason not to buy them, clean them properly (strong detergent, multiple washing, dry cleaning, whatever) and wear them. The only reason not to do so, is that it seems distasteful, and makes one squeamish to think of what might have happened in them in the past.
Having said that, I personally wouldn't do it...I know it's not rational, but I would feel icky putting them on.