I've never really felt the need. They don't really add much warmth, they add considerable weight (depending on what model you use) and don't really provide any real advantage.

There are some reasons-if for some reason you don't like the feel of your sleeping bag, (or for the quilt users out there), it can give a bit of protection.

I guess it does keep the bag slightly cleaner, but having used a sleeping bag for four months continually, and accumulation of body oils (or whatever the given reason is) was never really a problem.