I just do my biking in basketball shorts and carry a pair of casual shorts if I plan on getting off the bike. So when I stop there, I just put the shorts on over my basketball shorts and when I'm ready to ride again, I strip the casual shorts off.
Come to think of it, the only time I'm wearing lycra is on my "hardcore" road bike. And the only time I'm on it usually is on my organized rides or my training for an organized ride.