It is a bit of a complicated problem. These articles discuss the issue:
http://marginalrevolution.com/margin...st_publis.html
http://theness.com/neurologicablog/index.php/are-most-medical-studies-wrong/

The solution in part is to use more of a Bayesian analsyis, considering prior probability in doing research. As the authors of the above articles say:

1) In evaluating any study try to take into account the amount of background noise. That is, remember that the more hypotheses which are tested and the less selection which goes into choosing hypotheses the more likely it is that you are looking at noise.
2) Bigger samples are better. (But note that even big samples won't help to solve the problems of observational studies which is a whole other problem).
3) Small effects are to be distrusted.
4) Multiple sources and types of evidence are desirable.
5) Evaluate literatures not individual papers.
6) Trust empirical papers which test other people's theories more than empirical papers which test the author's theory.
7) As an editor or referee, don't reject papers that fail to reject the null.