An Esbit solid-fuel stove fits the bill for lightweight and budget:
http://www.rei.com/product/653343/esbit-pocket-stove
The Rivendell site says it better than I could: "It's steel, folds down to 3.9' x 3" x 3/4. It weighs 3.5 ounces with a fuel tablet that burns for 9 minutes, enough to boil some tea or soup. All in all, this is the perfect stove to bring when you're not sure you want to bring one, but aren't sure you want to do without, either. Because it's so cheap and small."
The main drawback is the fuel tablets are not readily available out in the countryside, so you would need to make sure you brought an ample supply.