Two things helped me out quite a lot. I switched to a Sun EZ Sport LWB recumbent and the seat has an adjustable lumbar support. I ride with a straight back and neck, relieving a lot of pinches &c.

Second, I bought a rowing machine that uses hydraulic cylinders for resistance. The cylinders allow you to sit on the machine backwards and push against the resistance. So, you have both pushing and pulling, which really keeps the core firmed up. It's a Kettler Favorit, if you are interested. A very well made thing, that. bk