I’ve also found p-hacking really interesting. I’d like to join discussion by bringing more problems around economic research papers, according to a paper I read. The working paper is titled ‘Transparency, Reproducibility, and the Credibility of Economics Research’, which discusses p-hacking and other troubles that cost credibility to economics research.
The main three problems of transparency found by the authors are: publication bias, specification searching, and inability to replicate results. The authors develop a model to predict if a research paper has false findings and also uses meta-analyses to test if certain papers have certain bias. One of the findings that surprised me the most is the fact in the tests of a wide collection of top publications, the authors found that 10 to 20% of significant results are misplaced.
The authors also review the transparency policies at top economics and finance journals, and measure the publication rates and rates of writing-up of results from experiments with strong,mixed, and null results. My favorite part of this article is the fact that it’s a great way of understanding the trends that are going in Economics regarding the best practices for transparency.
Replicability and data openness are becoming with more frequency a need in the field. The standards are being set on an all time high, which will cause more quality papers. I only hope the new trends don’t mean a set back for the produce of economic literature. We’ll see what the future brings for Economics research.
Christensen and Miguel. TRANSPARENCY, REPRODUCIBILITY, AND THE CREDIBILITY OF ECONOMICS