Avoiding Data Misuse: Case Studies on Invalid Data, Simpson’s Paradox, and Statistical Pitfalls

The rapid growth of the internet has ushered in a big‑data era where information is abundant but often riddled with circular citations and questionable reliability; this talk shares practical cases of data misuse and offers methods to avoid ineffective analysis.

Case 1: A medical blogger cites breakfast‑related health claims that originated from a 20th‑century cereal company advertisement, illustrating how definitions and selective evidence can distort truth.

Case 2: An advertisement for a toothpaste brand appears to show 80% dentist endorsement, yet the survey allowed multiple selections, meaning 100% also endorsed competing brands, highlighting partial data presentation.

Case 3: A 1995 UK safety warning claimed third‑generation contraceptives doubled the risk of blood clots; absolute risk increased from 15 to 25 per 100,000, a negligible change that was obscured by relative percentages.

Case 4: 2018 murder statistics suggested London’s homicide rate surpassed New York’s because only a few months were considered, while a full‑year view shows New York’s rate is far higher—an example of selective time‑frame bias.

Case 5: Comparing judicial decisions with algorithmic ones shows algorithms can reduce recidivism by up to 25% and lower detention by 40%, emphasizing the influence of subjective human judgment versus data‑driven models.

Case 6: A 1960s robbery investigation used conditional probability to illustrate how a seemingly tiny “innocent” probability (1/120,000) can be dramatically altered when the population context changes.

Case 7: Questionnaire reliability is examined through a loan‑offer scenario where 40% of users truly have better offers but 80% of those without will lie, inflating the affirmative response rate to 88% and distorting the perceived competitive gap.

Case 8: Airline delay data appears to favor one carrier due to Simpson’s paradox; aggregated lower delay rates mask higher rates on each individual airport, demonstrating the danger of ignoring subgroup composition.

Further discussions cover correlation vs. causation (e.g., beer container size and consumption), survivor bias in user retention analyses, and the importance of questioning data sources, sample representativeness, and the true purpose of any analysis.

To avoid invalid conclusions, the speaker recommends always challenging data provenance, ensuring analyses address concrete business problems, evaluating risk‑reward probabilities, and recognizing biases such as Simpson’s paradox, survivor bias, and causal misinterpretation.