How to Build and Forecast ARMA Models: A Step-by-Step Guide

Construction and Forecasting of ARMA Models

In practical modeling problems, the first step is model identification and order selection, i.e., determining the class of the model and estimating its order. This essentially reduces to the model order selection problem. Once the order is fixed, the model parameters must be estimated.

After order selection and parameter estimation are completed, the model must be tested, specifically checking whether the residuals constitute white noise. If the test passes, the ARMA time‑series modeling is considered complete. As an important application of time‑series modeling, the article also discusses forecasting with ARMA series.

AIC Criterion for ARMA Model Order Selection

The Akaike Information Criterion (AIC) was introduced by Japanese statistician Hirotugu Akaike in 1974. AIC is a key result of information theory and statistics.

The AIC order‑selection rule for a series is to choose the model that minimizes the AIC value, where AIC = -2 log‑likelihood + 2k, k being the number of estimated parameters and n the sample size. When the series contains an unknown mean parameter, the model includes a constant term; the number of unknown parameters becomes p + q + 1, and the same AIC formula is applied to select the model with the smallest AIC.

Parameter Estimation for ARMA Models

ARMA parameter estimation methods include method‑of‑moments, inverse‑function estimation, ordinary least squares, conditional least squares, and maximum likelihood. The mathematical derivations are omitted here; instead, Python libraries are used to obtain the parameter estimates directly.

Diagnostic Tests for ARMA Models

Let the residuals of the fitted model be denoted by e_t; they serve as estimates of white noise. For a given series, after estimating the parameters, the residuals are computed accordingly.

The Ljung‑Box test statistic is used to assess the residuals, where h denotes the number of autocorrelation lags considered. Under the null hypothesis of white noise, the statistic follows a chi‑square distribution with degrees of freedom equal to the number of estimated parameters. Given a significance level α, the critical chi‑square value is consulted; if the statistic exceeds this value, the null hypothesis is rejected, indicating the residuals are not white noise and the model fails the test. Otherwise, the null hypothesis is accepted, and the model passes the diagnostic check.