In statistics, a design matrix (also known as regressor matrix or model matrix) is a matrix of values of explanatory variables of a set of objects, often denoted by X. Each row represents an individual object, with the successive columns corresponding to the variables and their specific values for that object. The design matrix is used in certain statistical models, e.g., the general linear model. It can contain indicator variables (ones and zeros) that indicate group membership in an ANOVA, or it can contain values of continuous variables.
Contents
- Definition
- Simple Regression
- Multiple Regression
- One way ANOVA Cell Means Model
- One way ANOVA offset from reference group
- References
The design matrix contains data on the independent variables (also called explanatory variables) in statistical models which attempt to explain observed data on a response variable (often called a dependent variable) in terms of the explanatory variables. The theory relating to such models makes substantial use of matrix manipulations involving the design matrix: see for example linear regression. A notable feature of the concept of a design matrix is that it is able to represent a number of different experimental designs and statistical models, e.g., ANOVA, ANCOVA, and linear regression.
Definition
The design matrix is defined to be a matrix X such that the jth column of the ith row of X represents the value of the jth variable associated with the ith object.
A regression model which is a linear combination of the explanatory variables may therefore be represented via matrix multiplication as
where X is the design matrix,
Simple Regression
This section gives an example of simple linear regression—that is, regression with only a single explanatory variable—with seven observations. The seven data points are {yi, xi}, for i = 1, 2, …, 7. The simple linear regression model is
where
where the first column of ones in the design matrix allows estimation of the y-intercept while the second column contains the x-values associated with the corresponding y-values.
Multiple Regression
This section contains an example of multiple regression with two covariates (explanatory variables): w and x. Again suppose that the data consist of seven observations, and that for each observed value to be predicted (
This model can be written in matrix terms as
Here the 7×3 matrix on the right side is the design matrix.
One-way ANOVA (Cell Means Model)
This section contains an example with a one-way analysis of variance (ANOVA) with three groups and seven observations. The given data set has the first three observations belonging to the first group, the following two observations belonging to the second group and the final two observations belonging to the third group. If the model to be fit is just the mean of each group, then the model is
which can be written
It should be emphasized that in this model
One-way ANOVA (offset from reference group)
The ANOVA model could be equivalently written as each group parameter
with the constraint that
In this model