Blog Archives

Aside

The ‘file drawer problem’ was a term created Robert Rosenthal in 1979. It was a phrase composed to describe the numerous studies that may have been conducted but never reported. This problem is also known as publication bias, which is the tendency of researchers and editors to treat results that are positive (significant) differently to results that are negative (supporting the null hypothesis).

The extreme view of the ‘file drawer problem’  is that journals are filled with the 5% of the studies that show Type 1 errors (a false positive), while the file drawers back at the lab are filled with 95% of the studies that show non-significant results (p >.05). This means that some null hypotheses are in fact true and that the association being studied does not exist, but the 5% of studies that (by chance) show a statistically significant result are published instead (Rosenthal, 1979).

The ‘file drawer problem’ is a problem because false positive results are being published in professional journals. Effects that are not real may appear to be supported by research, thus causing serious amounts of bias throughout publicised literature (Bakan, 1967). The outcomes of some studies, for example a meta-analysis, rely heavily on published works that may have exaggerated outcomes. An additional issue is that investigators may waste precious time and effort conducting research on topics that have already been well-researched, but just have not been reported.

Although no definitive solution to this problem is available, estimations of damage to research conclusions can be made. (Rosenberg, 2005). The increasing interest of psychologists in summarizing entire research areas has lead to an improvement in bookkeeping. Rosenthal (1979) proposed a technique, based on probability, calculations for deciding whether a finding is resistant to the ‘file drawer effect’. This method is known as the fail-safe file drawer (FSFD) analysis. This analysis involves calculating a fail-safe number, which is then used to estimate whether or not the file-drawer problem is likely. Eventually, all results will be recorded with an estimate of effect size and with the level of significance obtained (Rosenthal, 1979).

However, Scargle (2000) has criticized Rosenthal’s method, stating that he fails to take into account the bias in the “file drawer” of unpublished studies, and thus can give confusing and misleading results. Scargle (2000) urges efforts, such as research registries, to try to limit publication bias.

It is important that the ‘file drawer problem’ is recognised and addressed in order for more psychologists to become aware. It appears that more researchers and reviewers of literature are taking into consideration the importance of null hypotheses. Hopefully, in the near future a solution for the ‘file drawer problem’ can be discovered.

References

Bakan, D. (1967) On method: toward a reconstruction of psychological investigation, 1st Edition, pp. 187

Rosenberg, M. S. (2005) The file-drawer problem revisited, Evolution, Vol 59 (2) pp. 464-468. DOI: 10.1111/j.0014-3820.2005.tb01004.x

Rosenthal, R. (1979) The “file drawer problem” and tolerance for null results, Psychological Bulletin, Vol. 86, No. 3, 838-641.

Scargle, J. (2000) Publication bias: The “file-drawer” problem in scientific inference,  Journal of Scientific Exploration, Vol. 14, No. 1, pp. 91-106

What is the ‘File Drawer Problem’ and why is it an issue?