Quantum‑Enhanced A3C² Leverages Time‑Series Dynamic Clustering for Adaptive ETF Stock Picking

Background

ETF passive replication typically rebalances quarterly or semi‑annually, which lags behind rapid market shifts in industry leadership, liquidity, and risk structure. Active fund managers, while flexible, often hold concentrated portfolios that degrade risk‑adjusted returns during volatile periods. Recent research applies machine learning and reinforcement learning to dynamically adjust portfolios, yet most clustering‑based strategies remain static and cannot track evolving market states. Advances in quantum machine learning (QML), especially variational quantum circuits (VQCs), provide high‑dimensional nonlinear embeddings that can improve representation power even on noisy intermediate‑scale quantum (NISQ) hardware.

Problem Definition

Pure A3C models are inefficient in high‑dimensional feature spaces and prone to over‑fitting.

Static clustering cannot guarantee that a cluster optimal in one period remains optimal in the next.

Method

Data and Feature Construction

The authors collect daily closing prices of all S&P 500 constituents from August 2021 to August 2024 for training, and from December 2024 to August 2025 for out‑of‑sample validation. For each stock i at time t, three features are extracted from the previous L days: 5‑day cumulative return m₅ᵢ(t), 20‑day cumulative return m₂₀ᵢ(t), and 20‑day return volatility v₂₀ᵢ(t). These capture short‑term momentum, medium‑term momentum, and local volatility.

Time‑Series Dynamic Clustering and Market Structure

At each decision window the feature matrix Φ(R₍t‑L:t₎) is clustered with K‑means, assigning each stock to one of K clusters. The mapping fₖₘₑₐₙₛ(·) produces a cluster‑level representation, reducing dimensionality from N′×3 to 4×K + 2. Each cluster Cₜᵏ is represented by the average 5‑day return m₅,ₜᵏ, average 20‑day return m₂₀,ₜᵏ, average 20‑day volatility v₂₀,ₜᵏ, and the proportion of stocks sizeₜᵏ.

State Representation

The agent’s state Sₜ at time t consists of the clustered market features together with the previous portfolio allocation. The exact mathematical formulation is shown in the following figure.

Action and Reward Design

At each step the agent observes Sₜ and generates a cluster‑selection policy via a quantum‑enhanced policy network that inserts a VQC as a nonlinear bottleneck. The log‑probabilities of actions are computed as:

For ablation, the VQC can be replaced by a parameter‑matched multilayer perceptron (MLP) while keeping the same dynamic‑clustering environment and A3C training setup. Actions are sampled from a temperature‑scaled softmax (τ = 1.3). The reward is a relative‑optimality measure that bounds rewards between +1 (best cluster) and a smooth quadratic penalty for sub‑optimal clusters:

This design aligns the monthly decision horizon with real ETF rebalancing schedules and mitigates non‑stationarity.

Experiments

Baseline and Ablation

Three baselines are compared: (1) a classic A3C with an MLP bottleneck (quantum‑free), (2) static K‑means clustering fitted once on the initial training window, and (3) a rolling‑clustering heuristic that greedily selects the cluster with the highest previous‑month return. All methods share the same monthly rebalancing timetable.

Training Stability and Reward Mechanism

Using Z‑score rewards caused the agent to over‑fit a few high‑feedback clusters, leading to unstable performance. The relative‑optimality reward produced stable cumulative rewards after roughly 2000 epochs, indicating balanced exploration‑exploitation.

Trading Performance

The model is trained for 4500 epochs on data from 2021‑08‑27 to 2024‑08‑31 and validated on 2024‑12 to 2025‑08. Compared with the S&P 500 benchmark, Q‑A3C² achieves a 17.09% cumulative return versus 7.09% for the benchmark. The active‑return curve shows sustained positive excess returns over many months, demonstrating the model’s ability to adapt to changing market structures.

Cluster Selection Dynamics

During the validation period the agent’s portfolio composition evolves: early months (Dec 2024–Jan 2025) focus on a few high‑volatility growth stocks such as TSLA and IBKR, reflecting a risk‑seeking mode. From Feb 2025 onward the number of selected stocks rises to 135, diversifying across information technology, healthcare, finance, and industrial sectors. Certain stocks (e.g., LULU, IBKR, CCL, UAL) appear repeatedly, indicating a learned preference for sectors aligned with macro‑economic cycles. In May 2025 the portfolio becomes highly concentrated on PLTR, likely driven by market volatility and AI‑related trends, then re‑diversifies in June toward utilities and defensive consumer staples, showing risk‑control behavior during potential market corrections.

Code example

m_{5, t}^{k}、m_{20, t}^{k}
和v_{20, t}^{k}分别是聚类C_{t}^{k}内的平均5天回报、平均20天回报和平均20天波动率，size_{t}^{k}是分配到该聚类的股票比例。
其中m5_{t}^{SPX}和m20_{t}^{SPX}分别表示标准普尔500指数所有股票的平均5天回报和20天回报。这样，每个聚类都以四个维度进行紧凑表示。