By reliable do you mean accurate? Any decent bike computer is going to work as long as you have it set up correctly and the battery is good. If you mean accuracy, how much do you really need? A computer that counts wheel rotations is going to be more accurate than GPS but if your ride is 21.3 vs 21.4 miles is that going to ruin your day? Probably not, right? Either one is good enough?

You said you want elevation change, and a cheap but quality bike computer won't do that in your price range. Personally I think elevation gain is more important (as data) than distance; it's easier to ride a flat mile than an uphill one. The elevation data adds context to the rest of what you have.

I'd recommend keeping your phone, maybe in your pocket. If you really want to spend $50 get a Polar H7 chest strap (HRM) and pair it with your phone and then have Strava record that, too.