Monthly Archives: February 2012

Aside

The ‘file drawer problem’ was a term created Robert Rosenthal in 1979. It was a phrase composed to describe the numerous studies that may have been conducted but never reported. This problem is also known as publication bias, which is the tendency of researchers and editors to treat results that are positive (significant) differently to results that are negative (supporting the null hypothesis).

The extreme view of the ‘file drawer problem’  is that journals are filled with the 5% of the studies that show Type 1 errors (a false positive), while the file drawers back at the lab are filled with 95% of the studies that show non-significant results (p >.05). This means that some null hypotheses are in fact true and that the association being studied does not exist, but the 5% of studies that (by chance) show a statistically significant result are published instead (Rosenthal, 1979).

The ‘file drawer problem’ is a problem because false positive results are being published in professional journals. Effects that are not real may appear to be supported by research, thus causing serious amounts of bias throughout publicised literature (Bakan, 1967). The outcomes of some studies, for example a meta-analysis, rely heavily on published works that may have exaggerated outcomes. An additional issue is that investigators may waste precious time and effort conducting research on topics that have already been well-researched, but just have not been reported.

Although no definitive solution to this problem is available, estimations of damage to research conclusions can be made. (Rosenberg, 2005). The increasing interest of psychologists in summarizing entire research areas has lead to an improvement in bookkeeping. Rosenthal (1979) proposed a technique, based on probability, calculations for deciding whether a finding is resistant to the ‘file drawer effect’. This method is known as the fail-safe file drawer (FSFD) analysis. This analysis involves calculating a fail-safe number, which is then used to estimate whether or not the file-drawer problem is likely. Eventually, all results will be recorded with an estimate of effect size and with the level of significance obtained (Rosenthal, 1979).

However, Scargle (2000) has criticized Rosenthal’s method, stating that he fails to take into account the bias in the “file drawer” of unpublished studies, and thus can give confusing and misleading results. Scargle (2000) urges efforts, such as research registries, to try to limit publication bias.

It is important that the ‘file drawer problem’ is recognised and addressed in order for more psychologists to become aware. It appears that more researchers and reviewers of literature are taking into consideration the importance of null hypotheses. Hopefully, in the near future a solution for the ‘file drawer problem’ can be discovered.

References

Bakan, D. (1967) On method: toward a reconstruction of psychological investigation, 1st Edition, pp. 187

Rosenberg, M. S. (2005) The file-drawer problem revisited, Evolution, Vol 59 (2) pp. 464-468. DOI: 10.1111/j.0014-3820.2005.tb01004.x

Rosenthal, R. (1979) The “file drawer problem” and tolerance for null results, Psychological Bulletin, Vol. 86, No. 3, 838-641.

Scargle, J. (2000) Publication bias: The “file-drawer” problem in scientific inference,  Journal of Scientific Exploration, Vol. 14, No. 1, pp. 91-106

What is the ‘File Drawer Problem’ and why is it an issue?

Advertisements

Does ‘False Consensus’ Effect Exist?

Standard

False Consensus Effect is an egotistic bias, causing one to believe that others in a group will respond to a situation in exactly the same way as oneself (Dawes, 1987). For example, assuming that people hold the same beliefs and opinions towards specific behaviours and attitudes.

One study into false consensus effect was conducted by Ross, Greene and House (1976), which provided evidence for the existence of the false consensus. In this study participants were asked to read about situations in which a conflict occurred, and then told two alternative ways of responding. They were then asked to do three things, 1. Guess which options other people would choose 2. Say which option they would choose, and 3. Describe the attributes of the person who would choose each of the two options.

The results from this study indicated more people assumed others would do the same as them, regardless of which of the two responses they chose themselves. This shows what Ross et al (1976) described as the ‘false consensus’ effect – the idea that we each think other people think in the same way we do, when actually they often do not.

However, Suls and Wan (1987) state that there are factors that influence the way we view others.  People possessing undesirable attributes overestimate consensus, whereas people holding desirable attributes underestimate consensus, with the latter pattern being a form of false uniqueness.

According to Marks and Miller (1987) almost every psychologist has, at some point, found it difficult to explain a study’s findings because they assume that others think in the same way as they do. For example, being unable to understand how someone could misinterpret an incredibly easy question.

In conclusion, it is clear that the false consensus effect exists, to some extent. For further conclusions to be drawn, more modern experiments need to be conducted. Many of the studies were carried out in the late 80’s, where it has been suggested most people thought less independently (Duffy et al, 1999). This means that in the 21st Century, people may be less prone to the false consensus effect.

 

References

 

Dawes, M. R. (1987) Journal of Experimental Social Psychology, Vol 25 (1) 1-17 doi: 10.1016/0022-1031(89)90036-X

Duffy, M. L., Jones, J., Thomas, S.W. (1999) Intervention in School and Clinic, Vol. 35 no. 1 34-37 doi: 10.1177/105345129903500106I

Marks, G. Miller, N. (1987) Ten years of research on the false-consensus effect: An empirical and theoretical review. Psychological Bulletin, Vol 102(1), 72-90. doi: 10.1037/0033-2909.102.1.72

Ross, L., Greene, D.,  House, P. (1976) Journal of Experimental Social Psychology, Vol 13, 279-301

Suls, J., Wan, C. K. (1987) Journal of Personality and Social Psychology, Vol 52 (1), 211-217. doi: 10.1037/0022-3514.52.1.211