Mastering ARMA: Build and Forecast Time Series Models with AIC and Python

Building and Forecasting ARMA Models

In practical modeling, the first step is model identification and order selection, i.e., determining the class of the model and estimating its order. This is essentially the model order‑selection problem. Once the order is fixed, the model parameters must be estimated.

After order and parameter estimation are completed, the model must be checked, i.e., testing whether the residuals constitute white noise. If the test passes, the ARMA time‑series model is considered complete. As an important application of time‑series modeling, the article also discusses forecasting with ARMA models.

ARMA Model Order Selection Using the AIC Criterion

The Akaike Information Criterion (AIC), proposed by Japanese statistician Hirotugu Akaike in 1974, is a key result of information theory and statistics.

For a series, the AIC order‑selection criterion chooses the order (p, q) that minimizes

AIC = n·ln(σ̂²) + 2·k, where n is the sample size, σ̂² is the estimated variance of the residuals, and k is the number of estimated parameters (including the mean if it is unknown). When the minimum is attained, the series is regarded as having the selected order.

If the series contains an unknown mean parameter, the model includes this mean, and the number of unknown parameters increases accordingly; the AIC criterion still selects the order that yields the smallest AIC value. In practice, the two formulations lead to the same minimizing order.

Parameter Estimation for ARMA Models

ARMA parameter estimation methods include method‑of‑moments, inverse‑function estimation, ordinary least squares, conditional least squares, and maximum likelihood. This article does not present the mathematical derivations; instead, it uses Python libraries to obtain the required estimates.

Model Diagnostic Tests

Let the residuals of the fitted model be e_t, which are estimates of the white‑noise process. For a given series, suppose the estimated parameters are θ̂; then the residuals are computed accordingly.

Here r denotes the lag at which the autocorrelation function cuts off, and the Ljung‑Box test statistic is

Q = n(n+2) Σ_{k=1}^{r} (ρ̂_k² / (n‑k)), where ρ̂_k is the sample autocorrelation at lag k.

The null hypothesis of the Ljung‑Box test is that the residuals are white noise.

When the null hypothesis holds and the sample size is sufficiently large, the statistic approximately follows a χ² distribution with degrees of freedom equal to the number of estimated model parameters. The test proceeds as follows: given a significance level α, look up the critical χ² value; if the statistic exceeds this value, reject the null hypothesis and conclude the residuals are not white noise (the model fails the diagnostic); otherwise, accept the null hypothesis and consider the model adequate.

References

司守奎，孙玺菁 Python数学实验与建模