I lifted weights for 4 years. First it was to lose weight. I dropped from 230 to 160, then I started putting on mass and at my high point I was your weight (185) but at 5'7 and under 10% bf.

Thanks to my mother's genetics my knees could not take lifting anymore so I switched to cycling. Now 6 months and 3000 miles later I've dropped down to 148. Almost 100% of the weight I have lost has been fat and upper body muscle mass.

Think about it. You're sitting on your bike burning calories and not using your upper body. Just eat healthy food, it's common sense, and the weight will come off naturally.

And if anything swimming will strengthen your upper body. It will not increase muscle mass. Plus that would be good cross training in the winter.