Bearings and tape, you can buy anywhere. I usually look on Amazon (I recently got a set of bearings that came in a range of sizes - 100 each - so I just got them to keep in stock).

For derailleurs, brake levers, etc, I've had the most luck with eBay. I rode out to my local bike co-op yesterday, but unfortunately, their used parts bins were very small. I was expecting to see all kinds of discarded vintage stuff, but found nothing.