Can you hear the noise or just feel it? If you can only feel it, then I'd say the seat has a broken seatpan or rail or seat post. Any of these things are potentially dangerous so you need to be sure that none of these things are broken.
If you can hear it as well as feel it, then it could still be the seat. But I'd say to look at the seat bag (if you have one) for it swinging and hitting the seat or seatpost. Check the contents of the bag too, sometimes wrenches will knock together and make all kinds of strange noises. (look at the tail light/rear reflector too if you have one for the same reasons)
Check the water bottle cages. I had a "squeek" that was driving me nuts last week. Turns out the seatpost water bottle cage was loose and moving when a full bottle was in place.
Check that the pump mount is securely holding the pump and that it isn't being hit by a crank arm/foot as you pedal.
Check the bottom bracket (crank "axle") for properly tightened cups. Check that the crank arms are tight.
Check the skewers are tight. If they are loose the wheel will shift in the frame dropouts and could make a noise and "bump" that you might feel.
Turn the bike upside down and carefully check ALL the frame tubes for cracks or long chips in the paint. Do the same when the bike is upright too. Cracked paint usually means the tube is cracked. (but not always)
My guess is that, because the noise only happens under rider weight in the saddle AND because you can feel it, the saddle is broken but DO NOT rule out the other possibilities.
Good luck and let us know what you find out.