Panniers.

I just leave the panniers on my bike at all times. They're cheap and run down so I've never had a theft problem. I actually put my bag inside the panniers while I ride.