External validity is the validity of generalized (causal) inferences in scientific research, usually based on experiments as experimental validity. In other words, it is the extent to which the results of a study can be generalized to other situations and to other people. Mathematical analysis of external validity concerns a determination of whether generalization across heterogeneous populations is feasible, and devising statistical and computational methods that produce valid generalizations.
Contents
Examples
If age is judged to be a major factor causing treatment effect to vary from individual to individual, then age differences between the sampled students and the general population would lead to a biased estimate of the average treatment effect in that population. Such bias can be corrected though by a simple re-weighing procedure: We take the age-specific effect in the student subpopulation and compute its average using the age distribution in the general population. This would give us an unbiased estimate of the average treatment effect in the population.
If, on the other hand, the relevant factor that distinguishes the study sample from the general population is in itself affected by the treatment, then a different re-weighing scheme need be invoked. Calling this factor Z, we again average the z-specific effect of X on Y in the experimental sample, but now we weigh it by the "causal effect" of X on Z. In other words, the new weight is the proportion of units attaining level Z=z had treatment X=x been administered to the entire population. This interventional probability, often written
A typical example of this nature occurs when Z is a mediator between the treatment and outcome, For instance, the treatment may be a cholesterol- reducing drug, Z may be cholesterol level, and Y life expectancy. Here, Z is both affected by the treatment and a major factor in determining the outcome, Y. Suppose that subjects selected for the experimental study tend to have higher cholesterol levels than is typical in the general population. To estimate the average effect of the drug on survival in the entire population, we first compute the z-specific treatment effect in the experimental study, and then average it using
The precise conditions ensuring the validity of this and other weighting schemes are formulated in Bareinboim and Pearl, 2016 and Bareinboim et al., 2014.
Qualitative research
Within the qualitative research paradigm, external validity is replaced by the concept of transferability. Transferability is the ability of research results to transfer to situations with similar parameters, populations and characteristics.
The basic dilemma of the social psychologist
When conducting experiments in psychology, some believe that there is always a trade-off between internal and external validity—
- having enough control over the situation to ensure that no extraneous variables are influencing the results and to randomly assign people to conditions, and
- ensuring that the results can be generalized to everyday life.
Some researchers believe that a good way to increase external validity is by conducting field experiments. In a field experiment, people's behavior is studied outside the laboratory, in its natural setting. A field experiment is identical in design to a laboratory experiment, except that it is conducted in a real-life setting. The participants in a field experiment are unaware that the events they experience are in fact an experiment. Some claim that the external validity of such an experiment is high because it is taking place in the real world, with real people who are more diverse than a typical university student sample. However, as real-world settings differ dramatically, findings in one real world setting may or may not generalize to another real world setting.
Neither internal nor external validity are captured in a single experiment. Social psychologists opt first for internal validity, conducting laboratory experiments in which people are randomly assigned to different conditions and all extraneous variables are controlled. Other social psychologists prefer external validity to control, conducting most of their research in field studies. And many do both. Taken together, both types of studies meet the requirements of the perfect experiment. Through replication, researchers can study a given research question with maximal internal and external validity.