In the UK, an early usage of the word is the more common—used by 'serious' roadies (cyclists who ride racing bicycles, and may themselves participate in competitive events) to refer to (stereotypically) bearded, sandal-wearing, cyclists without any high-tech gear. These Freds are not generally total novice cyclists, and often ride fairly frequently. This usage still survives in the US. David Bernstein, presenter of The FredCast says the term is "used by 'serious' roadies to disparage utility cyclists and touring riders, especially after these totally unfashionable 'freds' drop the 'serious' roadies on hills because the 'serious' guys were really posers."
(Wikipedia)
Sounds to me like a 'Fred' is just an ordinary cyclist who posers like to mock for not being what they deem to be a real cyclist on a 'real' bike. Just roll with it and see who's laughing when they can't keep up with you.