All you really need are shorts (most people will say bike shoes, too, and I use them--but but I ride most of the winter with platform pedals, toe clips and boots, and I don't go any slower, so i'm declaring bike shoes optional for those on tight budgets). I've ridden tens of thousands of miles over 40 years, including several centuries, in cotton T-shirts. They're comfortable for me, though many people say they get sweaty or abrade their nips. Gloves are helpful mainly by protecting your hands in a fall--i wear them, but if I were young and poor again, they'd be fairly low on my list.
I prefer mountain bike shorts to Lycra, and a fairly low-budget alternative is padded cycling underwear (Andiamo is one brand, and I think PearlIzumi makes some) that you can wear under street shorts.
Full disclosure: I feel like a jackass in full cycling gear. It's like putting on a football helmet and pads to play catch with your kids in the park.