This is what my bike looked like for a three-week trip on gravel roads in Idaho. I didn't use a backpack. I wouldn't call this ultra light however because my bike is rather heavy. A true ultralight setup would include a lightweight bike too. But that costs $$$.
Here's what my road bike looked like for about 10 days on the Pacific Coast Highway. Again no backpack and this time no stove.
I could've easily brought less. As mentioned earlier, the key is to simplify. Think about bulk, not weight. If you bring less stuff, and if the stuff you bring isn't bulky, you can use smaller bags to carry them, which means you don't need racks and panniers to support the bulk and weight. You can then use a non-traditional touring bike because your bike won't have such a big burden. Using a lighter bike with less carried weight means you can use lighter wheels and higher performance tires without the worry of breaking spokes. It really can snowball if you do it well.
In the backpacking world the official ultralight base weight (all gear except what you wear and what you consume) is 10lbs or less. I think it'd be reasonable to make a bike touring base weight of 15lbs or less as the ultralight standard. This would include all bags, racks, tools, spare parts, clothes carried, camping gear, cooking gear, gadgets, and other personal items that get carried by the bike. For some this would be an easy number to make. For others it would be a challenge. And for those of you who could care less, congratulations, you don't have to worry about it and can enjoy making fun of the geeks.