If it's only off by a little bit and it doesn't bug you then no, it's nothing to worry about too much. If you have a brake back there, you might want to get it addressed. Since you're worrying about chainline, I'm guessing this is a fixie or SS and likely doesn't have a rear brake, so...