The clipless is well worth it, biggest upgrade you can make.
Road shoes have a definite tres cool look and feel.

But sounds like you need the casual style MTB shoes. They flex more than carbon, but that's a good thing as it means they feel like a regular shoe when walking. Shimano and Pearl have them for well under $100.
You won't lose any speed with SPD over road. If you can't keep up with the group, it's not the shoes.