I sometimes ride through a part of town that is mostly poor and aboriginal to get to a pool that I swim at on weekends. People don't really notice me and many of them are also riding bikes in this neighbourhood. Just keep pedalling, be relaxed yet confident. If you look like you're minding your own business, they'll probably mind theirs.