Bike tests

Tire pressure is critical

If you test ride bikes, get the tires topped off with the same pressure. Don't just pull a bike off the rack and ride it. It's very hard to compare with wildly different pressures.

The front tire should be 10 psi to 15 psi lower than the back.
25c tires can use about 10 psi lower pressure, front and back.
The newer wide rims can drop it another 5 psi at least.

Some frames can now fit 28c tires. If your roads are very rough, I'd want those. You can run quite low pressures and not be any slower. (There's a slight aero disadvantage and a small weight penalty to larger tires, but they are still worth it.)

For example, I'm riding 23c tires on very wide 25mm rims, so the tires measure about 26.5 mm
I've been using 82 psi front, 95 rear, and I weigh 175 pounds.

Tires
The more expensive, thinner, flexible tires, like a Continental GP4000, will flex over rough roads better, for a smoother ride. So, cheaper, stiffer tires on the same bike won't feel as good. A lot of bikes come with these less expensive tires.

Frames

I find the carbon vibration advantage is fairly subtle, and tires make a bigger difference.
The carbon bikes are usually lighter, and may be stiffer for a little better handling and feeling more responsive.

I'd test both carbon and aluminum bikes.