(This article was originally published at Econometrics Beat: Dave Giles' Blog, and syndicated at StatsBlogs.)
Here's something that's very tempting, but it's not a good idea.
Using our sample of (n - 1) observations, the OLS estimator of β in (1) is:
b(n-1) = Σ (xi yi) / Σ(xi2) , (2)
where the summations in (2) run from i = 1 to i = (n - 1).
The predictor for yn is:
y*n = b(n-1) xn = xn Σ (xi yi) / Σ(xi2) . (3)
If we now estimate (1) using x1, x2, ...., xn; y1, y2, ...., yn-1, y*n, our OLS estimator of β will be:
b(n) = Σ (xi y'i) / Σ(xi2) , (4)
where y'i = yi for i = 1 to (n-1), and y'n = y*n. The summations in (4) run from i = 1 to i = n.
We can re-write (4) as:
b(n) = [Σ (xi yi) + xny*n] / [Σ(xi2) + xn2], (5)
where the summations in (5) run from i = 1 to i = (n -1).
Suppose that we want to estimate a regression model by OLS. We have a full sample of size n for the regressors, but one of the values for our dependent variable, y, isn't available. Rather than estimate the model using just the (n - 1) available data-points, you might think that it would be preferable to use all of the available data, and impute the missing value for y.
Fair enough, but what imputation method are you going to use?
For simplicity, and without any loss of generality, suppose that the model has a single regressor,
yi = β xi + εi , (1)
and it's the nth value of y that's missing. We have values for x1, x2, ...., xn; and for y1, y2, ...., yn-1.
Here's a great idea! OLS will give us the Best Linear Predictor of y, so why don't we just estimate (1) by OLS, using the available (n - 1) sample values for x and y; use this model (and xn) to get a predicted value (y*n) for yn; and then re-estimate the model with all n data-points: x1, x2, ...., xn; y1, y2, ...., yn-1, y*n.
Unfortunately, this is actually a waste of time. Let's see why.
Using our sample of (n - 1) observations, the OLS estimator of β in (1) is:
b(n-1) = Σ (xi yi) / Σ(xi2) , (2)
where the summations in (2) run from i = 1 to i = (n - 1).
The predictor for yn is:
y*n = b(n-1) xn = xn Σ (xi yi) / Σ(xi2) . (3)
If we now estimate (1) using x1, x2, ...., xn; y1, y2, ...., yn-1, y*n, our OLS estimator of β will be:
b(n) = Σ (xi y'i) / Σ(xi2) , (4)
where y'i = yi for i = 1 to (n-1), and y'n = y*n. The summations in (4) run from i = 1 to i = n.
We can re-write (4) as:
b(n) = [Σ (xi yi) + xny*n] / [Σ(xi2) + xn2], (5)
where the summations in (5) run from i = 1 to i = (n -1).
Substituting (3) into (5), and gathering up terms, we obtain:
b(n) = Σ (xi yi)[Σ(xi2) + xn2] / {Σ(xi2) [Σ(xi2) + xn2]}, (6)
b(n) = Σ (xi yi)[Σ(xi2) + xn2] / {Σ(xi2) [Σ(xi2) + xn2]}, (6)
where the summations in (6) run from i = 1 to i = (n -1).
Equation (6) reduces to:
b(n) = Σ (xi yi) / Σ(xi2) = b(n-1) .
In other words, nothing at all is gained by imputing the missing value of yn by using its OLS predicted value. You don't get something for nothing!
Equation (6) reduces to:
b(n) = Σ (xi yi) / Σ(xi2) = b(n-1) .
In other words, nothing at all is gained by imputing the missing value of yn by using its OLS predicted value. You don't get something for nothing!
Please comment on the article here: Econometrics Beat: Dave Giles' Blog
