I'm not sure how fast I am on the Brompton. . . I can imagine a few possible reasons for being slower that combined might add up to something: suspension bob, hub gear friction, chain tensioner friction, flexy frame. . . plus small wheels. Also, crap brakes make for longer stopping times.
But perhaps there's something mechanically up with your Brompton like hub cone adjustment?