Measurement invariance

Definition

In the common factor model, measurement invariance may be defined as the following equality:

f ( Y ∣ η , s ) = f ( Y ∣ η )

where f ( ⋅ ) is a distribution function, Y is an observed score, η is a factor score, and s denotes group membership (e.g., Caucasian=0, African American=1). Therefore, measurement invariance entails that given a subject's factor score, his or her observed score is not dependent on his or her group membership.

Types of invariance

Several different types of measurement invariance can be distinguished in the common factor model for continuous outcomes:

1) Equal form: The number of factors and the pattern of factor-indicator relationships are identical across groups. 2) Equal loadings: Factor loadings are equal across groups. 3) Equal intercepts: When observed scores are regressed on each factor, the intercepts are equal across groups. 4) Equal residual variances: The residual variances of the observed scores not accounted for by the factors are equal across groups.

The same typology can be generalized to the discrete outcomes case:

1) Equal form: The number of factors and the pattern of factor-indicator relationships are identical across groups. 2) Equal loadings: Factor loadings are equal across groups. 3) Equal thresholds: When observed scores are regressed on each factor, the thresholds are equal across groups. 4) Equal residual variances: The residual variances of the observed scores not accounted for by the factors are equal across groups.

Each of these conditions corresponds to a multiple-group confirmatory factor model with specific constraints. The tenability of each model can be tested statistically by using a likelihood ratio test or other indices of fit. Meaningful comparisons between groups usually require that all four conditions are met, which is known as strict measurement invariance. However, strict measurement invariance rarely holds in applied context. Usually, this is tested by sequentially introducing additional constraints starting from the equal form condition and eventually proceeding to the equal residuals condition if the fit of the model does not deteriorate in the meantime.

Tests for invariance

Although there is need for further research on the application of various invariance tests and their respective criteria across diverse testing conditions, two approaches are common among applied researchers. For each model being compared (e.g., Equal form, Equal Intercepts) a χ² fit statistic is iteratively estimated from the minimization of the difference between the model implied mean and covariance matrices and the observed mean and covariance matrices. As long as the models under comparison are nested, the difference between the χ² values and their respective degrees of freedom of any two CFA models of varying levels of invariance follows a χ² distribution (diff χ²) and as such, can be inspected for significance as an indication of whether increasingly restrictive models produce appreciable changes in model-data fit. However, there is some evidence the diff χ² is sensitive to factors unrelated to changes in invariance targeted constraints (e.g., sample size). As a result, researchers are also recommended to use the difference between the comparative fit index (ΔCFI) of two models specified to investigate measurement invariance. When the difference between the CFIs of two models of varying levels of measurement invariance (e.g., equal forms versus equal loadings) is greater than 0.01, then invariance in likely untenable. It is important to note that the CFI values being subtracted are expected to come from nested models as in the case of diff χ² testing; however, there is indication that applied researchers rarely take this into consideration when applying the CFI test.