I've been "car-free" for about 5 years now. Sometimes I miss not having one, but not that often. When I want to get out of the city for hiking or something, I just rent a car. Much depends on your area. Here in Seattle, it's fairly compact, making it much easier; everything I need to get to regularly is within about 4 miles of my residence (granted, there are lots of hills, but still). Also, public transit here is generally good.

I lived in SoCal (San Diego) for the first 14 years of my life. Since it's a more sprawling area, going car-free might be more difficult, but it's still do-able. If you live within 4 miles of the necessities you regularly need, it's certainly feasible.