Think about you and your bike as separate entities.
If YOU get robbed, have a backup $20+ on the bike. if the bike gets robbed, have a backup $20+ on your person. Enough that you can feed yourself until help arrives.
Any more than $100, in my opinion, and you're inviting trouble.