I have not looked into this from a gravel riding point of view, but I know with MTB wheels, the weight savings is only part of why people go with CF rims. The other is the difference in ride quality.

Whether the feel of CF rims is a plus or minus is apparently a personal preference thing.

And finally, CF rims can be pretty darn strong for a given weight, and never get bent/warped.