I suspect it would be deemed highly unpatriotic to squash you on July 4th when you are wearing the Red, White, and Blue. Yes, that would be especially heinous. I was riding the rural roads of E. Tennessee last year and thought a nice bright orange Vols jersey would have added to my security.
A good way to test that would be to ride around with a North Korean (Chinese???) flag jersey and see what happens. Maybe a jersey with a big picture of Fidel Castro or Karl Marx on the back?