In ridge regression, increasing the tuning parameter lambda strengthens the penalty and shrinks coefficients toward zero.

In ridge regression, increasing the tuning parameter lambda strengthens the penalty and shrinks coefficients toward zero.

• A. True
• B. False
• C. It depends on the data
• D. Only when lambda is negative

Answer: (A)

Ridge regression uses an L2 penalty that grows with the size of the coefficients. The objective combines the usual least-squares error with lambda times the sum of squared coefficients (excluding the intercept). When lambda increases, the penalty becomes more costly for large coefficients, so the solution is driven toward smaller values. Mathematically, the coefficients solve (X^T X + lambda I) beta = X^T y, and the added lambda I effectively dampens the coefficients, pulling them toward zero as lambda grows. This shrinkage reduces variance at the cost of some bias, which is the trade-off ridge regression is designed to manage. In practice, standardizing predictors helps the penalty act evenly across coefficients, since the penalty is scale-dependent. The intercept is typically not penalized, so it isn’t shrunk. A negative lambda would reverse the effect of the penalty, which is not how ridge regression is defined, and the described monotone shrinkage with lambda holds regardless of the data (though the exact amount of shrinkage varies with the data).

Ridge regression uses an L2 penalty that grows with the size of the coefficients. The objective combines the usual least-squares error with lambda times the sum of squared coefficients (excluding the intercept). When lambda increases, the penalty becomes more costly for large coefficients, so the solution is driven toward smaller values. Mathematically, the coefficients solve (X^T X + lambda I) beta = X^T y, and the added lambda I effectively dampens the coefficients, pulling them toward zero as lambda grows. This shrinkage reduces variance at the cost of some bias, which is the trade-off ridge regression is designed to manage. In practice, standardizing predictors helps the penalty act evenly across coefficients, since the penalty is scale-dependent. The intercept is typically not penalized, so it isn’t shrunk. A negative lambda would reverse the effect of the penalty, which is not how ridge regression is defined, and the described monotone shrinkage with lambda holds regardless of the data (though the exact amount of shrinkage varies with the data).