i am in south jersey...not sure about a big man group but there are slow groups around. How far north. I would ask at a good bike shop, mine refered me to various websites that had group rides.

I don't know them but google gave up this:
http://www.cjbc.org/
http://www.btcnj.com/

they both don't seem very fast/race oriented, for an easy check look at their classifications their A+ rides are only 18+ that is a C+ or so in a racey club.