If I was stuck with your budget, I'd go for a 26" wheeled mountain bike. Should be able to get it racked and panniered within your budget and have a couple bucks left over for some slick tires.

Of course, it's hardly the ideal touring bike. But you don't really get anywhere near ideal with a $400-$500 budget.