I've been down this path a few times. Plan to do laundry often. Pick a few color schemes and do a mix-and-match approach.
I would have two pair of business casual slacks (gray, khaki) and alternate those during the workweek.
One pair of blue jeans.
Embrace sweaters and stretchy clothes.

The thrift store near my house sells overruns as well as used clothing. Whenever a church fundraiser, corporate event, 5K race etc. had leftover t-shirts or polo shirts, they donated them to the thrifts.
And some of the used clothes donations had never been worn and had the original manufacturer's hangtags. I got a nice down jacket that way.
There are also dress-for-less stores with never-worn manufacturer/retail end-of-season items. I've picked up some very nice new fleece jackets for cheap there.

The real problem is not having baggy business suits. It's having a baggy birthday suit !