This is sad to hear that you get this reaction from people. I started cycling in 2014 as a big, fat guy and was always worried about rude comments or nasty remarks, but all I ever heard was people saying stuff like "Looking good" or "Keep it up".
People can really suck sometimes. Don't give up!