Bayesian experimental design provides a general probability-theoretical framework from which other theories on experimental design can be derived. It is based on Bayesian inference to interpret the observations/data acquired during the experiment. This allows accounting for both any prior knowledge on the parameters to be determined as well as uncertainties in observations.
Contents
- Linear theory
- Approximate normality
- Posterior distribution
- Mathematical formulation
- Gain in Shannon information as utility
- References
The theory of Bayesian experimental design is to a certain extent based on the theory for making optimal decisions under uncertainty. The aim when designing an experiment is to maximize the expected utility of the experiment outcome. The utility is most commonly defined in terms of a measure of the accuracy of the information provided by the experiment (e.g. the Shannon information or the negative variance), but may also involve factors such as the financial cost of performing the experiment. What will be the optimal experiment design depends on the particular utility criterion chosen.
Linear theory
If the model is linear, the prior probability density function (PDF) is homogeneous and observational errors are normally distributed, the theory simplifies to the classical optimal experimental design theory.
Approximate normality
In numerous publications on Bayesian experimental design, it is (often implicitly) assumed that all posterior PDFs will be approximately normal. This allows for the expected utility to be calculated using linear theory, averaging over the space of model parameters, an approach reviewed in Chaloner & Verdinelli (1995). Caution must however be taken when applying this method, since approximate normality of all possible posteriors is difficult to verify, even in cases of normal observational errors and uniform prior PDF.
Posterior distribution
Recently, increased computational resources allow inference of the posterior distribution of model parameters, which can directly be used for experiment design. Vanlier et al. (2012) proposed an approach that uses the posterior predictive distribution to assess the effect of new measurements on prediction uncertainty, while Liepe et al. (2013) suggest maximizing the mutual information between parameters, predictions and potential new experiments.
Mathematical formulation
Given a vector
where
The expected utility of an experiment with design
where
Gain in Shannon information as utility
Utility may be defined as the prior-posterior gain in Shannon information
Note also that
the Kullback–Leibler divergence of the prior from the posterior distribution. Lindley (1956) noted that the expected utility will then be coordinate-independent and can be written in two forms
of which the latter can be evaluated without the need for evaluating individual posterior PDFs
the expected information gain being exactly the mutual information between the parameter θ and the observation y. Kelly (1956) also derived just such a utility function for a gambler seeking to profit maximally from side information in a horse race; Kelly's situation is identical to the foregoing, with the side information, or "private wire" taking the place of the experiment.