While it technically IS theft, what you did...
If I came back to my bike and found that the wheels, pedals, seat, derailleur, etc were gone, I wouldn't give a ***** about the grips.
I say, if you can still live with yourself, well, live it up.