You will want brakes. I've done long tours on a fixie and brakes will save your ass. Think about the fact that you will need some spare clothes rain gear, camping stuff, cookware, water, and food. Basically your bike will be 40 pounds heavier. Try bringing that to a stop down a big hill without brakes. I toured with just a front brake and there where times where I wished I had two brakes. You might want a smaller gearing if you're riding in hilly areas.
There's nothing wrong or crazy about touring on a fixed gear. It's been done many times. In the 50's my grandpa went on tour on a fixed gear! But yeah, brakes, get them.