Structured Components-based Neural Network (SCNN) for Multivariate Time Series Forecasting: Theory, Implementation, and Business Application

Background Multivariate time series (MTS) forecasting is essential for business tasks such as order‑volume and user‑growth prediction, where data often contain complex spatial‑temporal structures that need accurate modeling.

Existing Methods and Limitations Traditional approaches like simple ARIMA or Bayesian Structural Time Series (BSTS) either ignore structural patterns or provide interval estimates, making them unsuitable for point‑wise predictions required by many business scenarios.

Model Introduction The Structured Components-based Neural Network (SCNN) decomposes an MTS into four modules: long‑term, short‑term, seasonal, and co‑evolving components. Each module is modeled separately and later fused via a convolutional neural network to produce the final forecast.

Multivariate Sequence Generation The original series \(Y_{n,t}\) is split into the four components using sliding‑window statistics (mean and variance) for long‑term and seasonal terms, and attention‑based weighting for the co‑evolving term. Equations and illustrative figures describe the decomposition process.

Structure Decomposition • Long‑term: large sliding window, mean and variance extraction. • Seasonal: similar window based on known period length. • Short‑term: shorter window to capture rapid fluctuations. • Co‑evolving: attention mechanism computes pairwise relevance between variables, producing normalized scores.

Structure Prediction Long‑term and seasonal components are forecast by directly using the most recent window’s mean. The short‑term, co‑evolving, and residual components are predicted with a shared autoregressive model: y_{t+1}=W\cdot y_t + b_i , where \(W\) and \(b_i\) are learned parameters.

Structure Fusion Predicted outputs of the four modules are concatenated and passed through a dual‑branch neural network (feature‑learning branch and parameter‑learning branch). Their outputs are merged by a 1‑D convolution to obtain the final point estimate.

Loss Function and Residual Handling The loss combines a mean‑likelihood‑estimation (MLE) term with a regularization term that penalizes large fluctuations. Additionally, a zero‑residual branch and a full‑residual branch are trained separately; their predictions are weighted to form the final objective, improving robustness.

Code Implementation The following Python snippets illustrate key parts of SCNN:

# Seasonal normalization
def SeasonalNorm(x, period_length):
    mean = F.pad(mean.reshape(b * c, n, -1), mode='circular', pad=(t % period_length, 0)).reshape(b, c, n, -1)
    var = F.pad(var.reshape(b * c, n, -1), mode='circular', pad=(t % period_length, 0)).reshape(b, c, n, -1)
    out = (x - mean) / (var + epsilon) ** 0.5
    return out, mean, var ** 0.5

# Co‑evolving normalization
class AdaSpatialNorm(nn.Module):
    def forward(self, x):
        mean_f = torch.matmul(adj_mat, x_f)
        var_f = torch.matmul(adj_mat, x_f ** 2) - mean_f ** 2 + 0.00001
        mean = mean_f.view(b, t, n, c).permute(0, 3, 2, 1)
        var = var_f.view(b, t, n, c).permute(0, 3, 2, 1)
        out = (x - mean) / (var + epsilon) ** 0.5
        return out, mean, var ** 0.5

# Long/short term normalization
def TermNorm(x, term_length):
    mean = F.pad(mean.reshape(b * c, n, -1), mode='replicate', pad=(term_length-1, 0)).reshape(b, c, n, -1)
    var = F.pad(var.reshape(b * c, n, -1), mode='replicate', pad=(term_length-1, 0)).reshape(b, c, n, -1)
    out = (x - mean) / (var + epsilon) ** 0.5
    return out, mean, var ** 0.5

# Fusion module
class SCNN(nn.Module):
    def forward(self, input):
        pred_distr_args = self.proj_distr_args(out)  # without residual
        pred_distr_args = (pred_distr_args[0], pred_distr_args[1], F.threshold(pred_distr_args[2], 0.2, 0.2))
        pred_distr = self.distr_output.distribution(pred_distr_args)
        pred_distr_args_aux = self.proj_distr_args(out_aux)  # with residual zeroed
        pred_distr_args_aux = (pred_distr_args_aux[0], pred_distr_args_aux[1], F.threshold(pred_distr_args_aux[2], 0.2, 0.2))
        pred_distr_aux = self.distr_output.distribution(pred_distr_args_aux)
        pred_mean = pred_distr_args[1].reshape(b, n, self.num_pred, 1).permute(0, 2, 1, 3)
        if self.training:
            return pred_distr, pred_distr_aux
        else:
            return pred_mean

Model Selection and Parameters Training/validation/test splits respect temporal continuity (e.g., days 1‑80 for training, 81‑120 for validation). Models are evaluated with RMSE and MAE; the best configuration (hidden channels, kernel size, etc.) is retained. Parameter groups include SCNN‑specific weights, generic neural‑network weights, training hyper‑parameters, and step‑size settings.

Business Scenario Application The model was applied to Ctrip’s OTA order‑volume data, using hourly steps (24‑hour seasonal period, 168‑hour long‑term period). Separate models were trained for peak (holiday) and off‑peak periods, then deployed to forecast March‑June 2024. Results showed average relative errors of ~1.3‑1.4% for daily volume, with larger deviations during holidays.

Future Improvements Planned enhancements include: incorporating prediction error into the loss for online self‑adjustment, building a platform that auto‑detects suitable variables for SCNN, enriching holiday training data with external signals (search trends, marketing activities), and developing dedicated holiday‑specific SCNN models.

Conclusion SCNN effectively disentangles multivariate time series into interpretable components, leverages neural‑network fusion, and achieves accurate point forecasts, demonstrating strong potential for both operational forecasting and causal‑impact analysis.

References

1. J. Deng et al., “Disentangling Structured Components: Towards Adaptive, Interpretable and Scalable Time Series Forecasting,” IEEE Transactions on Knowledge and Data Engineering, 2024. 2. SCNN source code: GitHub - JLDeng/SCNN 3. Q. Jiu et al., “Multivariate Bayesian Structural Time Series Model,” JMLR, 2018. 4. K. H. Brodersen et al., “Inferring Causal Impact Using Bayesian Structural Time‑Series Models,” Annals of Applied Statistics, 2015. 5. CausalImpact library: http://google.github.io/CausalImpact/CausalImpact.html