Bosh - I'm also went thru the "what do I need" phase in the past year. I found quite a few folks (and one LBS) that said I "had to have" a long list of gear. Not convinced of that I found a better LBS and did a lot of reading of these forums, Bicycling magazine, etc. My humble opinion (which I won't pretend is expert level):
Helmet - legally required in most places and really just a good idea with zero downside
Gloves - by your third or fourth ride (less on flat bar bikes) you'll realize how important they are. Get a good padded set and take care of them. Huge improvement in cycling enjoyment.
Bottle rack & bottle - you can probably ride an hour without drinking and be parched at the end. Putting on a rack and bottle makes short rides more enjoyable and mid range rides possible.
Tire pump - get a good standing pump with a pressure gauge. You'll use it.
And let's pause there. In my opinion you could get by with just the above equipment until you get out of the beginner stage. I did that for about 5 months of riding because I wanted to see if I'd stick with it before spending more cash. I used standard platform pedals, lightweight Merril hiking sneakers, regular workout gear (in bright colors) etc. I wasn't impressing anyone but I didn't care. I made sure I had a phone with me on all rides and someone knew I was riding. When I had one chain incident, I called someone to get me.
Once you get to the point where you commit, then you can add other gear. This is where a good LBS can be really helpful. Yes, you can save a few bucks on Amazon but the world is better if you spend your money locally. This is when you'll want to add a bag with spare tube, tire levers, patch kit, CO2 inflator, multi-tool, etc so you can do longer or unsupported rides. Adding clothing is all personal preference and riding style from there, although I think a high visibility jacket or shirt is a great start. Going further into padded bottoms, GPS units, upgraded saddles, etc are all great if your riding habits need them.
My favorite idea was to use the "things" as milestones, e.g. I wouldn't get my first cycling jacket until I could do a 10 mile ride in a certain time. Great motivational tool.
Get the basics, don't obsess, progress at your own pace and enjoy every little victory. Good luck!