Move because it is otherwise better place for you to live, but never just because its further from work.
You may not always be able to ride* and the option to get there with a 15min brisk, 20min normal walk is good to have.
*such as break an arm or both primary and secondary bikes are in deep repair, etc.
Al