Hard to tell since only you can interpret his tone of voice and how it came across. But it sounds to me like it was a simple off-handed remark. Maybe he was envious of you enjoying yourself and wished he was out riding too. Might have made the remark as sort of a "be careful out there buddy!"