i don't know about a few kJ, but it's probably about 2ish total.

Of course weight matters. That's not the issue. The question is about what is the optimal combo of weight vs aero. IE, what's the tipping point. Here in the states, in a P/1/2 crit, you're definitely going to see more of the better riders on pretty deepish aero tubulars than on box rims. But aero's clearly not the whole story, because you're also not going to see anyone on a disc rear either..