I don't think there is anything wrong with finding a little time on family vacations to enjoy your personal hobbies. Especially if you utilize down time when people are sleeping/ lounging or are busy with their own personal things. When I am on vacation the time I spend biking or surfing etc. makes me feel good and makes me more fun and energetic during the time I am spending with my family. I am a believer in a balanced life, family time is crucial but a bit of time for hobbies is healthy as well.
If your family members are into the same activities as you then by all means include them, the scenario Machka described above is a good example