OP, what is the make & model of your saddle? Two things I can think of; 1) wrong saddle, 2) saddle is too high. I've never had a problem with abnormal wear on an upright city bike. Same goes for millions of people in Europe who ride to work every day in chino's, cotton suit pants, and everything else imaginable. A good leather brooks is one option but you'll see few of these in Europe since replacing them after their stolen gets expensive. Here are a few saddle that will work well without wearing your pants out: Saddles. There are others.

If your seat is too high that will cause premature wear as well. On a city bike you should be able to just tiptoe the ground while sitting on your saddle. This is a bit lower relative to the pedals than you'd be setup for racing the Tour but works well for around town or 50 mile recreational rides on Saturday.