Depends what kind of racing you're doing. If you aren't contending for a podium then it might not be worth it. But if you already have the wheels I can't see any reason not to use them.

You can use less expensive 30mm aluminum wheels that get you 90% of the benefits of you Enve wheels but they'll have similar issues in the wind.

One comment on riding in a group. It's not correct that you lose the benefits of a deep wheel when drafting.