that's not 2cm off, that's 1cm from your 54.

and i doubt it will be a big deal. your speed will be a little slower but you'll have better control of your bike, which i'd prefer unless you are racing it. and it doesn't sound like you are. but if you just don't feel good getting the 53, then you should wait since it's something you'll have for a while. get it perfect if you can.