What is WLS (weighted least squares)

Weighted Least Squares (WLS)

Understanding WLS

Weighted Least Squares (WLS) is a statistical method used to fit a regression line to a set of data points when the variance of the error term is not constant across all observations. This condition is known as heteroscedasticity. Unlike Ordinary Least Squares (OLS), which assigns equal weight to all observations, WLS assigns different weights to different observations based on their variances.

The Core Idea

• Heteroscedasticity: This refers to the situation where the variability of the error term differs across observations. In simpler terms, the spread of the data points around the regression line is not consistent.
• Weighting: WLS addresses this by assigning weights to each observation. Observations with higher variance (less reliable) are given lower weights, while observations with lower variance (more reliable) are given higher weights.
• Minimizing Weighted Residuals: WLS aims to minimize the weighted sum of squared residuals, rather than the unweighted sum of squared residuals as in OLS.

Mathematical Formulation

Let's consider a linear regression model:

y = Xβ + ε

where:

• y is the dependent variable vector
• X is the design matrix
• β is the parameter vector
• ε is the error term vector

In WLS, the objective is to minimize the following weighted sum of squared residuals:

min β (y - Xβ)'W(y - Xβ)

where:

• W is a diagonal weight matrix with elements w_i corresponding to the weight of the i-th observation.

The optimal solution for β is given by:

β_WLS = (X'WX)^(-1)X'Wy

Interpretation

• Weights: The weights w_i are typically chosen to be inversely proportional to the variance of the corresponding error term.
• Effect on Estimates: By giving more weight to observations with smaller variances, WLS produces more reliable parameter estimates compared to OLS.
• Heteroscedasticity Correction: WLS is a common method to address heteroscedasticity in regression analysis.

Applications

WLS is used in various fields, including:

• Economics: Modeling economic relationships with heteroscedastic errors.
• Finance: Analyzing financial data with varying levels of uncertainty.
• Engineering: Fitting models to data with measurement errors of different precision.

Conclusion

Weighted Least Squares is a valuable tool for regression analysis when dealing with heteroscedastic data. By appropriately weighting observations, WLS improves the accuracy and reliability of parameter estimates compared to OLS.