Two step M estimators involving MLE

Two-step M-estimator involving Maximum Likelihood Estimator is a special case of general two-step M-estimator. Thus, consistency and asymptotic normality of the estimator follows from the general result on two-step M-estimators. Yet, when the first step estimation is MLE, under some assumptions, two-step M-estimator is more efficient [i.e. has smaller asymptotic variance] than M-estimator with known first-step parameter

Let {V_i,W_i,Z_i}n
i=1 be a random sample and the second-step M-estimator θ ^ is the following:

θ ^ ≔ a r g max θ ∈ Θ ∑ i m ( v i , w i , z i : θ , γ ^ )

where γ ^ is the parameter estimated by ML procedure in the first step. For the MLE,

γ ^ ≔ a r g max γ ∈ Γ ∑ i log ⁡ f ( v i t : z i , γ )

where f is the conditional density of V given Z. Now, suppose that given Z, V is conditionally independent of W. This assumption is called conditional independence assumption or selection on observables. Intuitively, this condition means that Z is a good predictor of V so that once conditioned on Z, V has no systematic dependence on W. Under the conditional independence assumption, the asymptotic variance of the two-step estimator is:

E[∇_θ s(θ₀,γ₀)]⁻¹ E[g(θ₀,γ₀ )g(θ₀,γ₀ )']E[∇_θ s(θ₀,γ₀)]⁻¹

where g(θ,γ) ≔ s(θ,γ)-E[ s(θ , γ) ∇_γ d(γ)^' ]E[∇_γ d(γ) ∇_γ d(γ)^' ]⁻¹ d(γ),

s(θ,γ) ≔ ∇_θ m(V, W, Z: θ, γ) , d(γ) ≔ ∇_γ log f (V : Z, γ), and ∇ represents partial derivative with respect to a row vector. In the case where γ₀ is known, the asymptotic variance is E[∇_θ s(θ₀,γ₀)]⁻¹ E[s(θ₀,γ₀ )s(θ₀,γ₀ )^']E[∇_θ s(θ₀,γ₀)]⁻¹ and therefore, unless E[ s(θ, γ) ∇_γ d(γ)^' ]=0, the two-step M-estimator is more efficient than the usual M-estimator. This fact suggests that even when γ₀ is known a priori, there is efficiency gain by estimating γ by MLE. An application of this result can be found, for example, in treatment effect estimation.