I think it mostly depends on what riding you're doing. for me, 30+ miles and hilly really necessitates at least toe clips and padded shorts. in defense of clipless, it really makes you feel like you're one with the bike (like fixie riders claim often...come to think of it, I wonder how clipless + fixie would work...)

However, 90% of the time, I'll keep my toe clips or ride without. Toe clips look good and fit most of my (classier looking) shoes; for short rides around town, I usually just go without any clippage and ride in sandals.

If it's for longer rides, yeah - you will feel the difference with clipless.