I think that's right.

If both riders are coasting (assumptions like this make physics much easier), the forces acting on them will be gravity pushing them down (which is proportional to their mass) and air resistance pushing them back (which is independent of their mass but increases with velocity). At some point, the force from air resistance will equal the force from gravity, and the rider will no longer accelerate without pedaling (terminal velocity). The heavier rider will require a greater force from air resistance to reach terminal velocity, which means if they had similar aerodynamics, he'd be going faster when he reached terminal velocity.

I think the difference in aerodynamics is less significant. It's also more complicated. The heavier rider, while having a larger cross section, may also be more round/less flat. I'm not sure how it would work out.