Thanks for the responses, guys. I live in a hilly area so my speed varies from 5mph up steep grades to 40mph coming down them. My average on flats, when I can find them, is around 18. Road quality is very good but I'm starting to do some event rides where it will probably be more variable.
What I think I'm hearing is that the difference is actually really minimal and it probably makes more sense to stick with 28s.