I guess I was "worried" about the same thing a few weeks ago when I asked about upper body strength loss and riding a bent. After riding a bent for about three weeks, and still doing my normal weight lifting and swimming, I haven't notice no loss in strength. In fact, I went UP in my bench by 15 pounds! Maybe I was over doing the weights.
Add some core exercises. I like this
site and do this three times a week. OK, I've only done it three times a weel once! I normally do it twice a week because I do a back to back set and cry when I breath too hard!
Then add some body weight upper body exercises like push ups, dips, and pull ups. If you have some dumb bells (even just one) add pull overs, bent over rows, curls, and extensions.
I did these exercises when I was riding my DF bike as well.
Jerry