I don't think it's rude if your other option is sitting in front of the TV and drinking.

I've already informed my family that I'm bringing comfortable shoes and will be going for a walk after the meal. A couple of them have said they want to join me.