In least squares regression, the LOOCV estimate for the test MSE can be calculated by fitting a model once.

In least squares regression, the LOOCV estimate for the test MSE can be calculated by fitting a model once.

• A. False
• B. Only for special cases
• C. True
• D. Never

Answer: (C)

In least squares regression with squared error, you can get the LOOCV estimate of the test MSE from a single fit because the influence of leaving out each observation has a closed-form expression in terms of the full fit. After fitting the model to all data, you have the residuals e_i = y_i − ŷ_i and the leverages h_ii from the hat matrix H = X(X'X)^{-1}X'. The LOOCV residual for observation i is e_i divided by (1 − h_ii), so the LOOCV estimate of the MSE is the average of these squared values: (1/n) ∑ [e_i / (1 − h_ii)]^2. Since this uses quantities from the single fit, you don’t need to refit the model n times. This shortcut is a well-known property of linear models with squared loss; for other models or losses, the shortcut generally doesn’t apply.

In least squares regression with squared error, you can get the LOOCV estimate of the test MSE from a single fit because the influence of leaving out each observation has a closed-form expression in terms of the full fit. After fitting the model to all data, you have the residuals e_i = y_i − ŷ_i and the leverages h_ii from the hat matrix H = X(X'X)^{-1}X'. The LOOCV residual for observation i is e_i divided by (1 − h_ii), so the LOOCV estimate of the MSE is the average of these squared values: (1/n) ∑ [e_i / (1 − h_ii)]^2. Since this uses quantities from the single fit, you don’t need to refit the model n times. This shortcut is a well-known property of linear models with squared loss; for other models or losses, the shortcut generally doesn’t apply.