In statistics, the g-prior is an objective prior for the regression coefficients of a multiple regression. It was introduced by Arnold Zellner. It is a key tool in Bayes and empirical Bayes variable selection.
Consider a data set
(
x
1
,
y
1
)
,
…
,
(
x
n
,
y
n
)
, where the
x
i
are Euclidean vectors and the
y
i
are scalars. The multiple regression model is formulated as
y
i
=
x
i
⊤
β
+
ε
i
.
where the
ε
i
are random errors. Zellner's g-prior for
β
is a multivariate normal distribution with covariance matrix proportional to the inverse Fisher information matrix for
β
.
Assume the
ε
i
are iid normal with zero mean and variance
ψ
−
1
. Let
X
be the matrix with
i
th row equal to
x
i
⊤
. Then the g-prior for
β
is the multivariate normal distribution with prior mean a hyperparameter
β
0
and covariance matrix proportional to
ψ
(
X
⊤
X
)
−
1
, i.e.,
β
|
ψ
∼
MVN
[
β
0
,
g
ψ
(
X
⊤
X
)
−
1
]
.
where g is a positive scalar parameter.
The posterior distribution of
β
is given as
β
|
ψ
,
x
,
y
∼
MVN
[
q
β
^
+
(
1
−
q
)
β
0
,
q
ψ
(
X
⊤
X
)
−
1
]
.
where
q
=
g
/
(
1
+
g
)
and
β
^
=
(
X
⊤
X
)
−
1
X
⊤
y
.
is the maximum likelihood (least squares) estimator of
β
. The vector of regression coefficients
β
can be estimated by its posterior mean under the g-prior, i.e., as the weighted average of the maximum likelihood estimator and
β
0
,
β
~
=
q
β
^
+
(
1
−
q
)
β
0
.
Clearly, as g →∞, the posterior mean converges to the maximum likelihood estimator.
Estimation of g is slightly less straightforward than estimation of
β
. A variety of methods have been proposed, including Bayes and empirical Bayes estimators.
