Mastering Grouped and Dummy Variable Regression: Weighted Models Explained

Grouped Data Regression

Not all data points are alike; for example, larger schools tend to have smaller variance than smaller schools, which is a case of heteroskedasticity—variance of the dependent variable changes with the size of the feature variable.

Grouped data are common because of privacy constraints; governments and companies often release only aggregated statistics. Regression can still be applied by using weighted least squares, giving more weight to groups with larger sample sizes and lower variance.

Consider a dataset of workers with education years and log hourly wages. Running an ordinary regression on the ungrouped data yields a set of coefficients. If the data are then aggregated by education level, we are left with only ten points: the average log wage for each group and the group size.

Using smf.wls (weighted least squares) instead of ordinary least squares, we assign each aggregated point a weight equal to its group count. The estimated coefficient for education remains the same, but standard errors increase slightly because some variance information is lost.

Dummy Variable Regression

Dummy variables turn categorical factors into binary columns (0/1). One category is chosen as the base; the others are represented by separate dummy columns to avoid perfect multicollinearity.

For example, a gender variable with categories male, female, and other can be encoded as two dummies: female and other . If both dummies are 0, the observation is male.

In A/B testing or treatment‑effect analysis, the dummy coefficient measures the intercept shift, i.e., the mean difference between treated and untreated groups.

To illustrate, we estimate the effect of completing high school (12 years of education) on hourly wage. We create a treatment dummy T that equals 1 if education > 12 and 0 otherwise.

When T = 0, the predicted wage is the intercept (≈ 19.9). When T = 1, the prediction adds the dummy coefficient (≈ 4.9), giving a predicted wage of ≈ 24.84. Thus the dummy captures a mean increase of 4.9 dollars.

If we add IQ as an additional covariate, the dummy coefficient now represents the conditional effect of graduating high school while holding IQ constant. An interaction term between the dummy and IQ allows the treatment effect to vary with IQ, producing non‑parallel prediction lines.

When we replace the single education variable with a set of dummies—one for each education year—we obtain a fully non‑parametric model that simply computes the average wage for each education level. This approach removes any functional‑form assumptions but often reduces statistical significance because many parameters are estimated with limited data.

Key ideas: weight observations by group size and variance; regression naturally accommodates grouped anonymous data via weighted least squares; dummy regression offers a flexible way to model categorical treatments without imposing a specific functional form.

https://github.com/xieliaing/CausalInferenceIntro