Much of the answer will depend on your personality. I personally enjoy tinkering with stuff and making older stuff work better than when it was new, so I enjoy buying used bikes and working with them to make them what I want. I live in an area with a good used bike market, though, and parts are plentiful on the US eBay site and even on local used advertisement sites. I also have room to work on and store more than one bike, so I have a few older bikes that I ride regularly.

If I did not enjoy working on older bikes and/or if I had room for only one bike, then I'd probably have one newer one, and probably one the costs more money.

Neither method is right or wrong -- it will just suit you better or worse depending on what interests you.