Your arm problem seems like the main issue. I'd suggest finding a friend who will let you borrow an upright bike, like a hybrid. Take it for a ride. If your arm seems okay, take it for a longer ride. Ride it two or three days in a row. If it works for you, great. I'm only familiar with one lightweight, road, straight bar bike - the Specialized Sirrus. I think it's a high-quality road option for someone who doesn't like drop bars. I'm sure other companies make something similar.
If riding a straight bar bike is still hard on your arm, how about a recumbent? A friend who rides one raves about the fact that he never has to deal with sore wrists, sore forearms, sore neck, etc.
If you're not sure whether you'll really take to bicycling, a $600 price limit may preclude some of these options. However, if you get into it for a long time, spending more will be worth it. Once you have the bike, there aren't a lot of expenses; you can ride it for many years without doing much to it except maybe buying tubes, new tires every few years, maybe replacing bar tape that gets frayed. My oldest bike is about 25 years old, and I still ride it.
Of course, if you get into cycling you can always find things to spend money on. My bikes haven't "worn out", but I've still felt a need to replace them. I'm presently building up my 5th bike.