Manipulation check

Manipulation check is a term in experimental research in the social sciences which refers to certain kinds of secondary evaluations of an experiment.

Overview

Manipulation checks are measured variables that show what the manipulated variables concurrently affect besides the dependent variable of interest.

In experiments, an experimenter manipulates some aspect of a process or task and randomly assigns subjects to different levels of the manipulation ("experimental conditions"). The experimenter then observes whether variation in the manipulated variables cause differences in the dependent variable. Manipulation checks are targeted at variables beside the dependent variable of interest.

Manipulations are not intended to verify that the manipulated factor caused variation in the dependent variable. This is verified by random assignment, manipulation before measurement of the dependent variable, and statistical tests of effect of the manipulated variable on the dependent variable. Thus, a failed manipulation check does not refute the hypothesis that the manipulation caused variation in the dependent variable.

In contrast, a successful manipulation check can help an experimenter rule out reasons that a manipulation may have failed to influence a dependent variable. When a manipulation creates significant differences between experimental conditions in both (1) the dependent variable and (2) the measured manipulation check variable, the interpretation is that (1) the manipulation "causes" variation in the dependent variable (the "effect") and (2) the manipulation also explains variation in some other, more theoretically obvious measured variable that it is expected to concurrently influence, which assists in interpreting the "cause" (i.e., it only help interpret the "cause"; it is not necessary to affirm that the "cause" causes an effect).

See also

• Design of experiments
• Instructional manipulation check