Lots of advice here already
I think the little that I could add is:
If anyone could go car-free with a child it would be you! Kudos on thinking through the issue so thoroughly already! Since the car is paid for, and in good shape, the only savings you would get by going car-free would be the insurance/registration/taxes(?).
You could join one of the car-sharing services - insurance is included. And you can always call a taxi for those occasional "must go" right now times.
But on the other hand ----
It is always good practice when contemplating major changes to do them serially rather than all at once. I heard that nothing changes your life like having children and that first time parents just have no idea what is going to happen.
So how about this?
Keep the car and just enjoy being car lite++++ for the time being. tsl brings up a very good point about the tyke needing to grow a bit before he/she can be carried about as "baggage".
In a year, with the okay from your pediatrician, you can revisit the idea. You will have lost nothing really by keeping the car and you will be able to make a more informed decision then.