Cycling need not be expensive. I started with a 20-year-old used bike, and it was more than good enough for the job. Of course it had only 12 gears, and down tube shifters, but it worked well enough for me. I bought some rather inexpensive Canali shorts and jerseys, a Gyro helmet, and I upgraded to Shimano SPD clipless pedals and cheap Lake cycling shoes. Altogether my initial outlay was under $500.
I made upgrades as I went along, many were for comfort reasons. The Lake shoes were the first to go; I would get hot spots on the bottoms of my feet during long rides, and sometimes the shoes would pop off the pedals. I upgraded to Look pedals and higher grade Shimano shoes. The Canali shorts were okay for rides under 50 miles, but after a year of riding I was doing 300 miles or more per week. I tried different brands of shorts, and found Assos Sportsline to be the most comfortable.
As for the bike itself, I was going to upgrade it to use Campy Ergo-power shifters/levers, but someone saved me the effort by stealing my bike. I maxes out my credit card and bought a Colnago Master Light with a Campy Record 9 speed group, and Mavic Helium wheels (top-shelf items at the time). It's been 13 years since I bought this bike, and I still ride it today.
Personally speaking, I dislike flat bars or riser bars. They are okay for going to the grocery store, and a quick commute to work, but drop bars offer more positions for your hands, and allow for a better riding position. Your seat should not be supporting your full weight, your hands and arms should carry a good share as well.
Speaking more of comfort, it is important that your bike fit. A good fitting bike will be much more pleasurable to ride. Find a good shop to fit your bike to you. Some of my local shops are helpless when it comes to fitting a bike. They have the hardware, but often don't know how to use it properly.
Don't fixate too much on the bike or clothes. Riding should be fun regardless of what you ride. I did a century ride on an old and rusty 1955 Schwinn cruiser. I didn't go very fast, but I had a great time, and was able to enjoy the scenery more than usual.
Find any excuse you can to ride, and never make excuses not to ride. At the time I was doing all of my heavy riding, I was working in a stressful position at a finance company, doing at least twelve hours a day. Riding burned off the stress, and gave me a good way to enjoy life even more.