I remove the loose labels but leave the one that require work to remove because i'm lazy. As long as huge chunks aren't popping off you're fine.