I'm 67 and 5'9" and find a 54 frame to be my max... but I suffer from short legs/long torso so smaller frames introduce other problems. Biggest surprise I read in your posts is that your shop did not do a comprehensive fitting when you bought the Felt; I have always assumed it was a standard to provide this free with a new purchase). I'd try hard to get them to help you dial in your fit. With 30 minutes on a trainer they should be able to diagnose your hand problems. I also have some hand numbness but I attribute it to multiple hand surgeries over the years; and I manage it with frequent position changes that drop bars give more than any other. And I also utilize progressive lens eyeglasses so I understand your head position/neck issues. All of which should be elements in a good fitting session.

When I returned to cycling in 2013 after a 35 year hiatus I was drawn by the totally new look and feel of the bikes and the incredibly different ride characteristics. The twitchy front end had both great liveliness as well as tremendous fear and loathing. A few hundred miles of conditioning made most of that go away. Fine tuning saddle height and fore/aft positioning from my initial fit helped immensely. And the constant search for the next tweek is, for me, part of the joy of the sport.

If I were you (and it seems I was for a while) I wouldn't give up on the Felt until you have exhausted your fit options. As many here have pointed out, there are huge changes possible in mere millimeters... a stem change of 10 or 15 mm can be huge. And changing bar height relative to saddle can be the difference between sadness and euphoria (a nod to Billy Joel). After a double metric century last summer my ride companion complained of his back and neck pain. I asked why he rode with such a long stem in the flattened position. He had no idea that his stem could be reversed. He has been in that euphoria place ever since.

Hang in there... Good luck.