With cool enough weather that would work, though may be uncomfortable if your ride is too long. You will probably need some form of ankle straps with that, unless your bike has a chain guard--you don't want clothing getting caught in the chain, it can cause a sudden deceleration!

These may not be big enough, 4X, but if you haven't looked at them, they would likely be a little more comfortable for longer rides.

http://www.aerotechdesigns.com/tights.htm#thermalpants