With rear panniers loaded with 40lbs of groceries (I was curious so I weighed them) my road bike gets squirrelly IF I stand up.
Not a problem if it wasn't also a fixed gear. Accelerating is difficult with a load but at speed it's fine. It does take a bit more to stop of course.