Mastering Model‑Centric Statistics: From Exploratory Analysis to Bayesian Inference

Model‑Centric Statistics

Statistics is primarily about collecting, organizing, analyzing, and interpreting data , so its fundamentals are essential for data analysis. Knowing how to write code in a language such as Python is a very useful skill for data preprocessing and analysis, even when data appear already tidy.

Exploratory Data Analysis

Data are the basic component of statistics, originating from experiments, computer models, surveys, and observations. If we are the data generators or collectors, we must first define the problem and the method before preparing the data .

In fact, statistics has a branch called experimental design that studies how to obtain data. In the era of data abundance, we sometimes forget that data acquisition is not always cheap. For example, a large hadron collider can generate hundreds of terabytes per day, yet its construction takes years of labor and intellect. This article assumes we have already obtained a clean dataset—a rare situation in practice.

Given a dataset, the usual approach is to explore and visualize it first, gaining an intuitive understanding. The exploratory data analysis (EDA) process can be summarized in two steps:

Descriptive statistics;

Data visualization.

Descriptive statistics uses metrics such as mean, mode, standard deviation, and interquartile range to quantitatively summarize data .

Data visualization presents data in vivid forms such as histograms and scatter plots . Although EDA may seem like a preliminary step before complex analysis, it remains valuable for understanding, interpreting, checking, summarizing, and communicating Bayesian analysis results.

Statistical Inference

Sometimes simple plots and calculations (e.g., computing the mean) suffice. Other times, we aim to extract more general conclusions from data—understanding how data are generated, predicting future unseen observations, or selecting the most plausible explanation among many . This is the purpose of statistical inference.

Models come in many forms; statistical inference relies on probabilistic models. Much scientific research and our understanding of the world are model‑based; the brain itself can be seen as a machine that models reality, as discussed in related TED talks.

A model is a simplified description of a system or process, focusing on important aspects rather than explaining the entire system .

If two models explain the same data with comparable performance, the simpler one is usually preferred—this is the Occam's razor principle .

Regardless of the model type, model building follows similar basic guidelines. The construction of Bayesian models can be summarized in three steps:

Given data and assumptions about how the data were generated, construct a (often rough) model.

Apply Bayesian theory to combine data and model, deriving logical conclusions—this yields a data‑fitted model.

Evaluate the model’s fit using various criteria, including real data and prior knowledge about the research question.

Reference:

Osvaldo Martin, Python Bayesian Analysis

Recommended books: