If it's in the same place of every down stroke, regardless of what gear you're in (and you've eliminated anything that may be hitting a crank arm), then (in my experience) it's likely your BB.