I'm not saying the lenses are big, I've handled a g1 and an oly pen and they are quite small for an interchangeable lens camera- it's just that in order to get the same range as my lx5 with an f2.0 it will still be more than I want to carry (and spend).
Travelling I'm happier w/ an all in one solution. And if I'm carrying a larger camera I want an optical viewfinder, and the shallower dof of a larger sensor. And affordable wide angle lenses. My Oly 420 is okay, but it's not a very good fit for me. All of the things that bug me about it have been carried over to micro 4/3.

Everyone's needs are different, and what they're willing to carry is different too. I'm coming from film photography, and back then I carried my yashica t4 with me everwhere. My slr, although not that big, never got taken on vacation. For the OP, I agree that micro 4/3 would likely be a good alternative.