It'll flex.

Is it square taper BB? If so, the crank will eventually move inwards on the spindle as the spindle / crank interface wears.
You could try a spacer or two in between your cog and the hub, then mount your chainring on the outside of your crank.
Is this a conversion?