Memory-Gated Recurrent Networks for Multivariate Time Series Analysis

1. Research Motivation

Multivariate time series consist of several interrelated variables evolving over time and appear in many domains such as e‑commerce sales, financial markets, and clinical monitoring. Existing machine‑learning models often ignore the complex dependencies between variables, treating all series uniformly and relying on the network to discover interactions, which is inefficient.

To address this, the authors propose a new recurrent architecture, the Memory‑Gated Recurrent Network (mGRN), that explicitly extracts two types of memory: marginal memory for each variable (or variable group) and joint memory for inter‑variable interactions.

2. Model Structure

The mGRN first partitions the M variables of a multivariate series into K groups. For each group, a marginal‑memory component, implemented as a GRU, captures the group’s own temporal dynamics (see Figure 1, red part). The joint‑memory component then aggregates the group representations to model their interactions (Figure 1, blue part).

Formally, the marginal‑memory component follows the standard GRU equations (see Formula 1). The key design is the element‑wise multiplication of the input and its memory, which aligns data and memory vectors explicitly, simplifying the network’s extraction of high‑dimensional dependencies.

The joint‑memory component combines the group outputs through a simplified GRU‑like update (Formula 2). To avoid over‑parameterization, the authors use GRU instead of LSTM and prune redundant gates in the joint‑memory part.

Two practical considerations are discussed: (1) variable grouping can be treated as a hyper‑parameter, with per‑variable grouping often yielding strong results; (2) the simplified mGRN can be easily combined with other architectures such as CNNs or attention mechanisms for further improvements.

3. Applications

mGRN is evaluated on several real‑world multivariate time‑series tasks:

Predicting patient outcomes in intensive care units (Harutyunyan et al., 2019).

Spoken digit recognition using multi‑frequency audio signals (Bagnall et al., 2018).

Handwritten digit trajectory classification (Bagnall et al., 2018).

In addition, a financial case study predicts short‑term stock price direction from limit‑order‑book data. The order‑book is split into four groups (buy price, buy volume, sell price, sell volume) and fed to mGRN. Experiments on Chinese A‑share stocks (CSI300/CSI500) from 2014‑2017 show that mGRN consistently outperforms LSTM and GRU in both accuracy and AUC (see Figures 2‑3 and Table 1).

4. Conclusion

The proposed Memory‑Gated Recurrent Network effectively separates marginal and joint memories in multivariate time series, reducing the difficulty of learning high‑dimensional temporal dependencies. Across multiple domains, mGRN achieves notable and stable improvements over existing recurrent models, highlighting its potential as a versatile tool for complex sequential data analysis.

References

[1] Zhang, Y.; Wu, Q.; Peng, N.; Dai, M.; Zhang, J.; Wang, H. (2021). AAAI‑21, arXiv:2012.13121. [2] Bagnall, A.; Dau, H. A.; Lines, J.; Flynn, M.; Large, J.; Bostrom, A.; Southam, P.; Keogh, E. (2018). UEA multivariate time series classification archive, arXiv:1811.00075. [3] Harutyunyan, H.; Khachatrian, H.; Kale, D. C.; Ver Steeg, G.; Galstyan, A. (2019). Multitask learning and benchmarking with clinical time series data, Scientific Data 6:1‑18. [4] Sirignano, J.; Cont, R. (2019). Universal features of price formation in financial markets, Quantitative Finance.