Don't just sit over the bumps. Use a little pedal pressure to take the weight off your backside, or stand up. Just taking your weight on your feet for a moment can make all the difference.

The style of bike matters. What do you have? If you're on a cruiser that has you sitting bolt upright, then any shock goes right up through your spine. In that case, some sort of sprung saddle might be worth a look.

The shape of the saddle can matter too. For whatever reason, I favor flatter saddles. Ones having a rounded shape from side-to-side cause me pain. Do you have any cycling friends with spare saddles? If you can borrow a few, it pays to experiment.