OK, so over the past two years we made the following HVAC upgrades to our 4300 sf house (3200 sf on main two floors, 1100 sf finished walkout basement office):
• Replaced the two traditional in the wall 10,000 BTU A/C units in the basement office (they leaked cold air like crazy in winter) with a single 12,000 BTU mini split (this allowed the re-insulation, sheet rocking/sealing of the walls over those old A/C units).
• Replaced the 12 year old contractor grade boiler (i.e. not very efficient) with a state of the art Energy Kinetics System 2000 boiler/hot water system (it's about the most efficient oil boiler on the market today)
• Replaced the 12 year old aging/leaking A/C unit with an Armstrong 16 SEER A/C/Heat Pump. Also upgraded the main ducting insulation in the attic from R4 to R8+ (they wrapped R8 AROUND the old R4).
The cost of all this was about $22K. The A/C unit wasn't really so optional, it was on its way out with a leak that might have needed a $500-$1500 repair on a relatively old central A/C unit.
We're nearing the end of the first full heating season with this new setup. When we started we were using 1500+ gallons of heating oil each winter. Looks like this winter we'll be @ 800 or so, give or take. At this point with the milder March temps, the heat pump is doing 80% of the heating, which really takes the load off the boiler. That's a 45%+ reduction. If we're saving 700 gallons annually @ $3 per gallon, this system's payback is over 10 years, but that could shorten up if oil prices rise further. That's not a lot of oil for a house this large (my old house in NJ c. 1926 construction consumed 1500 gallons per season for about 2300, about half the size).
Being able to turn the thermostat in the basement office down to 55º at night means we don't use the boiler to heat that space unless I'm down there working, and the sealed walls allow that space stay above 55º all night even on the coldest nights without that zone being turned on. It only takes 45 mins or so in the AM to warm back up.
The only other measure we haven't done is foam insulation in the attic, which might cut heat loss here and save another 150-200 gallons annually, but the cost of doing that is $4K, so the payback is hard to justify. But if oil goes to $5/gallon, we may look at that again. Other measures like solar powered hot water are pointless, the System 2000 boiler uses VERY little oil to generate the hot water.
Did a ton of research on this, looked into geothermal, could't make the numbers work, that install here would cost $35K or so, and that would have involved keeping the cranky old boiler for the basement and as a backup. We still would have used 400 gallons of oil annually to heat the basement zone and the hot water. Saving 400 gallons annually over what we wound up doing didn't make a lot of sense to me, particularly considering the greater cost and newer technology (if you think nothing can go wrong with a geothermal system, do some more research, it can become a major pain).
Ultimately I think geothermal is a total no brainer for new construction or major renovation. But in our retrofit application, what we did seems to have worked.