I had a similar problem. My crank broke. The bike was still ridable, but the right side crank (where the chain is) was wobbly.

So, yeah. I'd have it looked at.