Topo lines are going to miss a lot of elevation changes that your device with barometric sensor will record and accumulate. Topo lines at best are going to have a 10 foot vertical spacing. So it's possible to go up and down many times between them.

Most of the Garmin Edges with barometric altimeter are supposed to be able to resolve two feet of difference. So in the right terrain, I could have a gain loss of hundreds of feet and never cross a topo line.

The interval of contour maps as well as the resolution of elevation information in map database will always make comparing device information to map derived gain/loss the same as comparing apples to oranges.