Are you losing strength/muscle mass from your upper body or just fat? If you started with a low body fat % and aren't eating enough calories to fuel your bike rides then your body may be metabolizing your muscles to get the energy it needs. I'd start by figuring out how many calories you need to be eating to maintain your weight based on the amount of riding you do. Eating crap just to put some fat on your upper body seems like a bad idea.