Articles
Published:

Breaking the Qubit Barrier: Optimizing the S&P 1500 with Quantum Computing

We optimized a 932-asset S&P 1500 portfolio using 10 qubits to discover 256 organic market structures, breaking scaling limits that prevent quantum methods from being applied to real financial datasets.

The “Noisy Intermediate-Scale Quantum” (NISQ) era is often defined by constraints: shallow circuits, noisy qubits, no error correction, and limited connectivity. For finance, the most painful constraint is qubit count.

Most academic work solves toy problems with 5–10 assets. Real indices like the S&P 1500 contain thousands of liquid assets. To make quantum finance meaningful today, we need a way to process 1000+ variables using the 50–100 imperfect qubits available on modern hardware.

Based on recent work from Fujitsu, Large-scale portfolio optimization using Pauli Correlation Encoding, we pushed this and ran portfolio optimization on 932 assets using circuits of 10 qubits.

Using the Qoro stack: Divi’s modularity and extendibility and Qoro’s Cloud services, in a matter of hours we developed and scaled portfolio optimization to the largest gate-based execution that we know of.

Portfolio Optimization

In quantitative finance, Portfolio Optimization can be formulated as a Quadratic Unconstrained Binary Optimization (QUBO) problem. The goal is to maximize returns for a given risk appetite. However, mapping these problems to NISQ-era hardware presents two distinct physical blockers:

  1. Qubit Count: A portfolio of 500 assets requires a qubit volume exceeding most current QPUs. Traditional methods use a 1:1 mapping.
  2. Connectivity: Financial covariance matrices are dense. Mapping a fully connected graph to a superconducting processor (which typically has limited nearest-neighbor connectivity) requires swap networks that result in prohibitive circuit depths.

The standard approach to solving combinatorial optimization problems (like QAOA or VQE) is One-Hot Encoding. In this schema, every optimization variable maps to a single physical qubit.

  • 10 Assets – 10 Qubits. (Easy)
  • 50 Assets – 50 Qubits. (Hard, but possible).
  • 1000 Assets – 1000 Qubits. (Impossible on today’s gate-based hardware).

Even if we had a 1000-qubit device, the connectivity would be a showstopper. Financial correlation matrices are dense—everything correlates with everything. Implementing a dense graph on a linear quantum chip requires thousands of SWAP gates to move information around. This explodes the Circuit Depth, meaning the calculation takes so long that the quantum state collapses (decoheres) before we get an answer.

Attempt 1: Spectral Partitioning (why it failed)

Our first attempt to solve this used Spectral Partitioning. The idea was simple: use classical graph theory to cut the 1000-asset graph into smaller, manageable sub-graphs, and then solve those sub-graphs on the Quantum Computer. The ideas came directly from a recent paper from the JPMorgan team: Decomposition Pipeline for Large-Scale Portfolio Optimization with Applications to Near-Term Quantum Computing.

While this reduced the qubit count, it actually didn’t fix the circuit depth. When you force-partition a financial graph, the resulting clusters are extremely dense cliques. Translating these sub-problems into QUBO instances resulted in very deep circuits, with more than 10,000 two-qubit gates at 50-qubit partitions. Gate-based quantum computers can currently handle on the order of 100s of two-qubit gates which far exceeded the limit.

Partitioning reduced qubits, but increased depth so dramatically that the approach became unusable.

We needed a method that was Hardware Efficient—shallow circuits that respect the linear topology of the chip—while still handling massive datasets.

The Solution: Pauli Correlation Encoding

We turned to Pauli Correlation Encoding. Unlike the 1:1 mapping, PCE decouples the number of variables from the number of qubits. It allows us to represent M variables using approximately log_2(M) qubits.

PCE turns 1000 assets into 10 qubits by assigning each asset a ‘quantum barcode’ — a unique pattern of parities across qubits. Instead of representing a stock directly, we represent its relationship with every other stock.

Instead of Qubit 0 representing “Apple” and Qubit 1 representing “Google,” the state of the assets is determined by the joint parity of a subset of qubits. This Hamiltonian representation can be represented entirely in the Z basis, which means with observable grouping, we only need to run a single circuit per parameter choice.

This creates a massive solution space using very few resources. We then used a Hardware Efficient Ansatz with linear entanglement. This keeps the circuit depth constant and shallow (20-50), regardless of how many assets we add, and makes it perfect for running with matrix product state simulation. We apply just two layers of this for optimization.

Maximizing Hardware Throughput: Problem Packing

Fujitsu’s original method executes PCE in small batches — fine for statevector simulation, inefficient for cloud QPU use. This is because many optimization problems would have to run, leading to a lot of queuing and wasted qubit resources at small scale.

To scale this, we used optimization problem packing: multiple PCE optimization tasks combined into a larger block-diagonal structure executed on a 40-qubit system in parallel.

We created a Single-Shot Ensemble Pipeline:

  • Parallel Cuts: The quantum processor solves 4 market segmentation tasks at once.
  • Restricted Batched Execution: We repeat with different random seeds.
  • Fingerprinting: Assets that consistently land on the same side of each cut form highly correlated groups.

This produces 256 organic market clusters via quantum interference — a structure that is extremely difficult to obtain reliably with classical heuristics.

Divi: Pure Algorithmic Definition

To quickly put this together, we could leverage the modularity and the extendibility of our Divi software platform.

Standard quantum libraries often struggle when you need to inject custom decoding logic into the VQE optimization loop. Divi’s architecture allowed us to build a high-performance, problem-specific solver by subclassing its core components. It maintains it’s integration with our cloud platform, allowing us to quickly scale.

1. Custom VQE Implementation

We extended Divi’s base VQE class to implement the Single-Shot Ensemble logic. By overriding the post_process_results method, we bypassed standard Hamiltonian measurements and injected our optimized Bitwise Matrix Multiplication. This reduced the post-processing time for 17,500 shots from seconds to milliseconds.

quantum computing

2. Hardware-Efficient Ansatz

We used GenericLayerAnsatz to construct the circuit. Circuit depth is critical. We utilized a “Brick” entangling layout (linear nearest-neighbor), which is mathematically optimal for MPS (Matrix Product State) simulations. This allowed us to simulate 40+ qubits on standard computing hardware without the memory explosion associated with all-to-all connectivity.

3. Backend Agnosticism

One of the biggest challenges in quantum research is moving from a laptop simulator to a massive cloud simulator or QPU. Divi’s JobConfig allowed us to toggle between local debugging and high-performance execution with a single line:

  • Local Debugging: divi.backends.ParallelSimulator for fast, local CPU simulation.
  • Production Run: divi.backends.QoroService to dispatch the quantum jobs to the Qoro Maestro high-performance backend.

By leveraging Divi’s modularity, we transformed a theoretical clustering algorithm into a production-grade pipeline capable of processing the entire S&P 1500 in less than an hour.

The Experiment: Surviving the Crash

We set up a rigorous backtest using the S&P 1500:

  • Training Data (2022): We trained our Quantum Clustering algorithm on the 2022 Market Crash. The algorithm learned correlations during high volatility.
  • Testing Data (2023): We validated the portfolio during the 2023 “AI Recovery” Bull Market.

We implemented a Double Selection Strategy to construct the final portfolio from the quantum clusters:

  1. Quantum Defensive (The Safety Play): Selects the asset with the minimum volatility from each cluster. This aims to minimize drawdown during crashes.
  2. Quantum Aggressive (The Efficiency Play): Selects the asset with the highest historical Sharpe ratio from each cluster, balancing risk and reward.

We also tested with a “most momentum” strategy that chose assets based on relative strength, but found it generally was close to the aggressive model in our tests.

Results

The experiment processed 932 liquid assets and successfully identified 256 unique clusters. The results demonstrate that the quantum algorithm successfully disentangled the market structure, allowing us to engineer portfolios with radically different behaviors from the same dataset.

StrategyReturnRisk (Volatility)Sharpe Ratio
Benchmark (S&P Equal Weight)+22.67%0.17271.31
Quantum Defensive+11.23%0.14670.77
Quantum Aggressive+19.24%0.15161.27

Key Takeaways:

  1. Risk Reduction: The Quantum Defensive portfolio (Green line) achieved the lowest risk of all strategies (0.147 vs Benchmark 0.173), a ~15% reduction in volatility. This confirms the algorithm correctly identified “Safe Haven” clusters within the S&P 1500.
  2. Capturing the Upside: The Quantum Momentum portfolio (Blue line) successfully pivoted to growth, capturing nearly all of the benchmark’s upside (+19.2%) while still maintaining lower volatility than the broader index.
  3. Scale: Unlike toy examples that pick 4 or 5 stocks, this method constructed robust portfolios of 256 assets, making it a viable candidate for institutional risk management.

While the benchmark slightly outperformed in raw returns due to the heavy concentration of the “Magnificent 7” Tech giants in 2023, the Quantum approach succeeded in its primary goal: Engineering portfolios with specific, predictable risk characteristics using quantum correlations.

The Tensor Network Frontier: Why We Still Need Quantum Hardware

A natural question is: If we can simulate 40–50 qubits with MPS so efficiently, why do we still need real quantum hardware? The reason is that our current experiment intentionally used a hardware-efficient, low-entanglement ansatz. These circuits obey an area-law of entanglement, which makes them highly compressible and well-suited for tensor-network simulation. This is why our 40–50 qubit runs complete in seconds — and why this experiment represents the practical limit of what classical hardware can support.

However, real financial markets do not behave like 1D chains. They exhibit non-local, long-range correlations where distant assets influence one another. Modeling these effects requires moving to circular or all-to-all entangling layouts, as well as deeper, more expressive ansatzes. These produce volume-law entanglement, where MPS and other classical methods break down: memory requirements explode exponentially, and simulation becomes intractable.

This is where quantum processors become essential. A QPU can execute these highly entangled circuits with the same native cost as simple ones, enabling us to capture market structure far beyond the reach of classical simulation. Our current results show what is possible at the classical boundary; the next breakthroughs will come from deploying QPU-native ansatzes that reveal correlations no classical method can feasibly extract.

Why this matters

This work demonstrates that quantum-classical hybrid pipelines can scale today to real-world financial datasets without waiting for fault-tolerant devices. By combining hardware-efficient encoding with high-performance simulation, this approach represents a new path for deploying quantum methods in risk management, factor discovery, and large-scale portfolio construction.

To provide the best experience, we use technologies like cookies to store and/or access device information. You can manage your preferences here.