The Effect of Experimenter Demand on Inference,
K. Pun Winichakul, Guillermo Lezema, Priyoma Mustafi, Marissa Lepper, Alistair Wilson, David Danz, and Lise
Vesterlund, 2024.
In preparation for submission.
Abstract
PDF
To assess the threat of experimenter demand, we ask whether a hypothetical 'ill-intentioned'
researcher can manipulate inference.
Four classic behavioral comparative
statics are evaluated, and the potential for false inference is gauged by differentially
applying strong positive and negative experimenter demand across the relevant
decision pair. Across three different subject pools (laboratory, Prolific, and
MTurk), we find no evidence of experimenter demand eliminating or reversing directional
effects. The response to experimenter demand is very limited for all three
subject pools and is not large enough to generate false negatives, though we do find
evidence of false positives when testing precise nulls in larger online-subject pools.