I used to have my fixie set up like this. The only problem exists with the pedals, which have to go on the wrong sides if the cranks switch sides. The pedals do have the tendency to work themselves out of the cranks, but if you tighten them enough, this will not happen.
The only real problem I encountered was with the pedal "flip-up teeth", the little triangles you use to flip your toe into the clip. With my pedals backwards, these were on the front and not the back of the pedal, making them inaccessable.