Mastering Multiple Linear Regression: Models, ANOVA, and Dummy Variables

Multiple linear regression, also known as multiple regression, is a widely used regression method that involves several independent variables to model the linear relationship with a single dependent variable, allowing evaluation and prediction.

Multiple Linear Regression Model

The model uses multiple predictors to explain the response variable and can be estimated using the least‑squares method.

Example 1: An analyst regresses a company’s sales growth rate on GDP growth rate and sales staff growth rate, obtaining an intercept and slopes. Given the forecasted GDP growth and expected change in staff, the model predicts the sales growth rate.

Analysis of Variance (ANOVA)

Similar to simple regression, the ANOVA table for multiple regression provides the sum of squares, degrees of freedom, mean squares, and allows calculation of the coefficient of determination and the standard error of estimate.

Coefficient of Determination

The coefficient of determination (R²) equals regression sum of squares divided by total sum of squares. It indicates the proportion of variance in the dependent variable explained by all predictors. However, R² always increases as more predictors are added, regardless of their relevance.

Therefore, the adjusted R² is used, which does not necessarily increase with additional variables and can be lower than the unadjusted R², even negative.

Standard Error of Estimate

The standard error (SEE) equals the square root of the residual mean square. A smaller SEE indicates a better regression model.

Testing Regression Coefficients and Confidence Intervals

Hypothesis tests assess whether each coefficient equals a specific constant (often zero). The t‑test evaluates individual slopes, while the F‑test examines the joint hypothesis that all slopes are zero. Confidence intervals are constructed using the appropriate critical values.

Example 2: For a sample of 43 observations with a three‑variable regression, the F‑test at a given significance level shows the statistic falls in the rejection region, leading to rejection of the null hypothesis that all slopes are zero.

Dummy Variables

Qualitative predictors are incorporated as dummy variables (0 or 1). For a binary categorical variable, one dummy is sufficient; for a variable with k categories, k‑1 dummies are needed. Dummy variables allow testing for significant differences between categories.