I think it depends largely on where you live. Get an idea of how far apart things are.
It's doable.

I know people who regularly do it.

The people who probably know it best could be down at your LBS, at least in my experience. Do the owners have kids? They'd be interesting to talk to if you're looking for people who have a busy schedule (with kids) and are probably committed to going places by bike.

People who say you can't do it are the same type of people who say you can't ride to work, they have never tried it and probably never will.