At 20 miles it makes sense to wear some cycling-specific clothes. For me, that would be cycling shorts or liners underneath some cycling-friendly shorts, trousers, or knickers (capris for men). By cycling-friendly I mean offerings from Sugoi, Nashbar, Chrome, Rapha, Swrve (stuff from their urban/lifestyle/commuter product lines that have some subtle design elements geared towards cycling). On top, I would go with either a jersey or (more frequently) just a comfortable (but not too flappy) t-shirt (with some sort of cycling-friendly outer if weather warrants).

You can in theory wear other casual or athleisure wear but I find, for bottoms at least, the cycling-friendly options have meaningful design elements such as articulated knees, seamless crotches, etc., that provide better comfort and maybe better wear.