1. No stability concerns.
2. Performance shouldn't get too killed. Once your rack is fit properly to the bike, taking it on/off is a pretty quick and easy job. Wake up a bit early on the weekend and you could do it easily.
3. Frame won't have any problem. Besides, if you bought it new, your frame has a lifetime warranty.