My experience is that bow-hunters are some of the most careful hunters out there. They have to be dedicated and patient. They don't waste shots unless they are certain of what they are shooting at.
Once I biked into the middle of a bear hunt, and the hunters weren't bothered at all. They told me my riding through wouldn't make one whit of difference to their hunt.
I worry the most during rifle-deer season. That's when you tend to get the once-a-year hunters out there. Then I wear orange and other bright colors, trade my white helmet for a red one, and tend to stay on well-defined trails.
Hunters know that others are out in the woods. Good hunters won't hold it against you that you're out trying to enjoy the outdoors just like they are.