From Research Paper to Production Workflow: Building Hybrid Optimize

Personal retrospective: This article describes my own work and views. It is not an official Strangeworks publication or an endorsement by my former employer.

Hybrid Optimize began without a roadmap item, a customer specification, or a pre-existing service design.

After building Strangeworks' QAOA and VQE products, I was given the freedom to choose what I developed next. I came across Liu and Goan's paper, Hybrid Gate-Based and Annealing Quantum Computing for Large-Size Ising Problems, and became excited by the practical opportunity behind the mathematics.

The paper proposed the large-system sampling approximation, or LSSA: decompose a large Ising problem into smaller subsystems, solve those subsystems, and use a smaller gate-based variational calculation to determine how their solutions should contribute to an approximate solution of the full problem.

What caught my attention was not only the algorithm, but that Strangeworks was unusually well suited to turn the idea into a product. We already provided access to a distributed network of gate-based computers, annealers, quantum-inspired machines, simulators, and classical solvers. An algorithm that needed to coordinate different kinds of computational resources was a natural fit for the platform.

I proposed the product and designed and implemented it end to end: the object-oriented Python SDK, backend service, decomposition strategies, provider-independent child-job workflow, asynchronous Modal execution, result mapping, final quantum optimization, packaging, tests, deployment, documentation, and benchmarking.

This article is not a claim that I invented LSSA. It is a case study in a different kind of engineering work: recognizing when a research idea fits the capabilities of a platform, and then supplying everything the paper does not specify to make that idea usable as software.

At a glance: I independently proposed Hybrid Optimize, then designed and implemented its SDK, service, decomposition strategies, asynchronous parent/child workflow, solver-independent interfaces, final quantum stage, tests, deployment, documentation, and benchmarks. The research paper supplied the mathematical method; this case study concerns the work required to turn it into a product.

The boundary between the research and my work

Accurate attribution matters, particularly when discussing work that started from a research paper.

The paper supplied the central mathematical method. It described sampling subsystems from a larger Ising problem, solving them independently, encoding their solutions into a variational calculation, optimizing their amplitude contributions using the full-system Hamiltonian, and reconstructing an approximate global ground-state configuration.

It did not describe a production software architecture for a multi-provider platform.

I developed Hybrid Optimize as its sole implementation owner. I was constrained—in useful ways—by the existing Strangeworks platform, its product library, job model, resource model, and portal. I had also learned a great deal while discussing the earlier QAOA and VQE services with our CTO and chief architect. Within those boundaries, however, I chose the product direction and made the Hybrid Optimize design and implementation decisions myself.

That distinction is central to the case study. Senior technical work does not always mean inventing a new algorithm. It can mean understanding a research result well enough to identify its product potential, designing the system that can deliver it, and owning the consequences of that design in production.

A mathematical proposal is not yet a product

At a high level, LSSA sounds straightforward:

1. Split a problem into smaller problems.
2. Solve the smaller problems.
3. Combine their solutions.

Each of those steps conceals a software-design problem.

What model formats should users be allowed to submit? How should a graph be divided? How do we know which variable in a child result corresponds to which variable in the original problem? Which solver should run each subproblem? How do we represent a workflow containing dozens of asynchronously executing jobs? What happens when some have completed and others are still queued? How does the final stage find the correct input artifacts? How do users inspect the workflow, retrieve a result, or diagnose a failure?

The public Strangeworks Hybrid QAOA documentation shows the eventual user-facing result. The SDK accepted several common optimization representations, including NetworkX graphs, QUBO matrices, Qiskit operator objects, and dimod binary quadratic models. Users could select the subsystem-solving resource separately from the gate-based resource used for the final calculation.

The published strangeworks-hybrid-optimize package provides a second public record of that user-facing SDK. I link to it as evidence that the product was shipped, not as a substitute for the architectural account here.

As the wider Optimization product matured, I also aligned Hybrid Optimize with its model and submission conventions. I reused shared conversion types and helper methods—including the public strangeworks-optimization-models package—so that submitting a problem to Hybrid Optimize felt consistent with submitting it to the standard Optimization service. This reduced duplicated conversion logic and allowed improvements to common optimization models to flow into the hybrid product.

That interface deliberately hid most of the distributed workflow beneath it.

The SDK made the operation look like one job. The service had to make it behave like one.

The first parent/child job workflow

Hybrid Optimize was the first practical demonstration of Strangeworks' parent/child job functionality, later informing the design of our Strangeworks Optimization flagship product.

A single user request created a parent Hybrid Optimize job and dispatched the longer-running preparation work to a Modal function. The service could then return control to the user without making them wait for decomposition or for every child submission to finish. The Modal worker generated the subsystem definitions and submitted a child job for each one. Those jobs could remain queued or running for very different amounts of time. The parent had to remain inspectable while the child jobs progressed, and the platform portal needed to show the hierarchy rather than a disconnected collection of submissions.

This introduced a completion barrier into the workflow:

The difficult part was not sending many HTTP requests. It was maintaining a reliable relationship between four forms of state:

• the full input problem;
• the exact subgraph definition used by each child;
• the identity and lifecycle of the corresponding child job; and
• the solution that eventually came back from that job.

If any one of those associations was lost, the workflow could still produce a plausible-looking answer—just not an answer to the problem the user submitted. That makes variable and subgraph identity an architectural invariant rather than an implementation detail.

For every subsystem, I retained an explicit mapping from its compact local variable indices back to the variables in the full problem. When a child result arrived, the service used that mapping to place its values into the correct positions of a full-length vector. The final calculation therefore consumed structured evidence about the original problem, rather than an unordered collection of bitstrings.

Keeping expensive preparation out of the request path

Decomposition could itself be mathematically intensive. Depending on the requested strategy and problem size, the service might need to manipulate a large weighted graph or recursively partition it before any solver work could begin. After decomposition, creating many child jobs required a sequence of HTTP requests to other Strangeworks services.

Neither workload belonged on the synchronous API path. If the submission endpoint performed all of it before responding, the user would be forced to wait while the service carried out work whose duration grew with the problem. A sufficiently large workflow could also encounter an HTTP timeout after some child jobs had already been created, leaving the user uncertain about whether their submission existed.

I used Modal to establish an asynchronous boundary:

1. Validate the request and create the parent job.
2. Dispatch a Modal function responsible for decomposition and child submission.
3. Return the durable parent-job identity to the user.
4. Let the Modal worker create the subsystem manifest and submit each child.
5. Reconcile child statuses and continue to the final stage independently of the original request.

Modal therefore did not replace the child solvers or define the scientific workflow. Its role was to execute the potentially slow preparation and fan-out reliably outside the interactive request. This kept the SDK responsive while preserving a visible parent record for the work that had been accepted.

Decomposition was a product decision

The research method requires sampled subsystems, but a product still has to decide how to construct them.

I implemented and compared multiple strategies. A weighted-random method expanded a subsystem by favouring variables with strong interactions with the variables already selected. I also implemented recursive Kernighan–Lin bisection, using graph structure to divide the problem into bounded-size partitions. A simpler random strategy provided a useful baseline.

These strategies exposed real trade-offs:

• Larger subgraphs preserve more of the original problem context but require larger child devices.
• More subgraphs provide more evidence and greater coverage but create more jobs, cost, and orchestration overhead.
• Random sampling can increase diversity but may neglect structurally important interactions.
• Graph partitioning can preserve local structure while weakening information carried across partition boundaries.
• Overlap between subgraphs can help connect local answers, but only if repeated variables are mapped and combined consistently.

These are algorithmic decisions, but they are also product decisions. Users need parameters that express the trade-offs without requiring them to understand internal service boundaries or provider APIs. The SDK therefore described the workflow in terms of subsystem count, subsystem size, decomposition strategy, and algorithm parameters—not in terms of which internal endpoint happened to execute each stage.

Solver independence made the idea fit the platform

When I first built Hybrid Optimize, the Strangeworks Optimization service did not yet exist. The initial subsystem workflow called the QAOA service.

I designed the hybrid service so that the decomposition and orchestration logic was not intrinsically tied to one QAOA backend. After the Optimization service became available, I adapted Hybrid Optimize so that a subsystem could instead be sent through that service to a quantum annealer, quantum-inspired optimizer, or classical backend.

The resulting hierarchy could be deeper than it first appeared:

Hybrid Optimize parent
└── subsystem child in the Optimization service
    └── provider-execution child for the selected solver

This was more than swapping one API call for another. QAOA and optimization solvers represented models, parameters, statuses, and results differently. The Hybrid service needed a stable internal concept of a solved subsystem while allowing the selected child service to retain responsibility for provider-specific execution.

I increasingly reused the Optimization service's helper methods and its shared model package rather than maintaining a parallel set of conversions inside Hybrid Optimize. The goal was architectural as well as ergonomic: a user who already knew how to represent and submit a problem to Optimization should not have to learn a conflicting model language for Hybrid Optimize. The hybrid-specific SDK parameters described decomposition and final recombination; the underlying optimization problem remained expressed through familiar contracts.

That separation was one of the most important architectural choices:

• Hybrid Optimize owned decomposition, the subsystem manifest, workflow coordination, and reconstruction.
• The selected child service owned the mechanics of solving one subsystem.
• The SDK exposed one product-level workflow rather than leaking the internal service graph to the user.

It also meant the product could benefit from new solver integrations without embedding every provider directly into the Hybrid service.

Reconstructing a solution on a smaller gate-based device

Suppose the full problem contains $N_p$ binary variables. Hybrid Optimize samples $N_s$ subproblems, each containing at most $N_g$ variables. After the child jobs return, each subsystem solution is mapped into an $N_p$-dimensional classical vector. Variables that were not part of that subsystem carry no contribution from that vector.

The final gate-based calculation does not encode all $N_p$ original variables as qubits. Instead, it uses a smaller circuit to learn coefficients for the collection of subsystem solutions. With $N_f$ final-stage qubits, as many as $2^{N_f}$ subsystem solutions can be indexed in the circuit basis.

Conceptually, the service constructs a weighted classical vector

where the subsystem solution vectors have already been mapped back into the coordinates of the full problem. The parameterized quantum circuit determines the coefficients $C_i$. The classical optimizer evaluates those coefficients using the energy of the full Hamiltonian, not merely the sum of the child objectives.

The sign of the resulting contribution at each full-problem variable gives the approximate binary or spin assignment. Where the weighted evidence is too close to zero to make a confident assignment, the implementation can test a bounded number of alternative assignments and retain the one with the best full-problem energy.

The important systems point is that the final stage only works if the preceding orchestration has preserved the scientific meaning of every intermediate result. Job tracking, variable mapping, artifact handling, and the variational calculation are not separable concerns here: they form one correctness chain.

A 78-variable scale demonstration

I later applied Hybrid Optimize to the aircraft cargo-loading QUBO family described in the AWS Quantum Technologies case study, which ranges from 26 to 78 binary variables.

For the largest 78-variable instance, I configured Hybrid Optimize to:

• construct 64 subgraphs;
• limit each subgraph to 20 variables;
• solve those subgraphs independently;
• map all 64 results back into the 78-variable problem; and
• perform the final coefficient optimization on a gate-based quantum device.

Because 64 subsystem solutions can be indexed using six final-stage qubits, no single quantum stage needed to encode the original 78-variable problem directly. The largest subsystem circuits required 20 qubits.

In that internal experiment, the reconstructed solution achieved the same benchmark objective score as our direct 78-variable run. The result is useful as a demonstration of resource decomposition: a workflow constrained to much smaller quantum circuits reproduced the solution quality of the larger direct encoding for that instance.

It is not evidence of quantum advantage. This experiment was not intended to outperform a mature classical optimizer such as Gurobi, which provided the exact classical baseline in the public cargo-loading study. The Hybrid Optimize result showed how a larger problem could be studied with smaller quantum devices; it did not show that a quantum workflow could beat state-of-the-art classical optimization.

The Hybrid Optimize result is a historical internal measurement rather than a reproducible public benchmark. I therefore use it only to illustrate resource decomposition, not to make a comparative performance or cost claim.

Architectural impact

Hybrid Optimize became a specialist capability available to the Strangeworks customer team for appropriate projects. Its broader importance inside the engineering platform was as an early demonstration of several capabilities that became increasingly relevant:

• hierarchical parent/child jobs;
• coordination of heterogeneous solver services;
• a provider-independent SDK over a distributed workflow;
• model and result movement across service boundaries; and
• reconstruction of one product-level result from many asynchronous jobs.

What I would preserve—and what I would change

Several design choices proved durable:

• Keep the user-facing workflow independent of individual providers.
• Keep slow decomposition and multi-job submission outside the synchronous request path.
• Treat the local-to-global variable mapping as first-class data.
• Make child jobs visible and diagnosable beneath the parent.
• Let child services own provider execution while the Hybrid service owns scientific orchestration.
• Separate decomposition policy from solver selection.
• Reuse canonical optimization model types and conversions instead of creating service-local equivalents.
• Evaluate the reconstructed answer against the full problem.

If I were designing the service again today, I would make several operational ideas more explicit:

• Give every child submission an idempotency key and a durable attempt record.
• Make completion and partial-failure policies declarative.
• Offer an explicit degraded-completion mode when a small subset of child jobs fails. Because the decomposition deliberately creates overlap between subgraphs, the loss of one or two results may still leave adequate coverage of the original variables and interactions. Rather than discard the successful work from dozens—or potentially around a hundred—other solver runs, the service could show the missing coverage and let the user choose whether to proceed with the final optimization using the surviving subsystem solutions.
• Base that decision on coverage as well as the raw number of successful jobs. The service should identify which variables and interactions have lost supporting subgraphs, preserve the failed-child diagnostics, and label the final result as degraded so that continuing does not silently weaken its scientific meaning.
• Prefer event-driven reconciliation where the surrounding platform supports it, with scheduled polling as a recovery mechanism.
• Build a sanitized benchmark harness alongside the product so that architectural and performance claims remain reproducible after the original project has ended.

Those conclusions came partly from Hybrid Optimize itself and partly from later work integrating a much larger portfolio of optimization providers. The companion case study, Designing for the Solver You Have Not Met Yet, explores that later modular solver-orchestration problem in more detail.

What this project says about engineering leadership

The part of Hybrid Optimize I value most is not the quantity of code or the novelty of any one class.

I found a research result that aligned unusually well with the capabilities of our platform. I made the case—initially to myself—that it should exist as a product. I learned the method deeply enough to implement its mathematical workflow, but I also identified everything the paper left unspecified: API design, model normalization, service ownership, asynchronous execution, subgraph identity, child-job visibility, provider selection, error handling, artifacts, results, and deployment.

I then built and operated the whole path.

That combination is the kind of work I want to continue doing in a senior engineering, architecture, or product-leadership role: moving between research, product judgement, and production systems without treating them as separate worlds.

Architecture in this setting is not a diagram drawn before implementation. It is the set of decisions that keeps a scientific idea correct while it passes through real software, real infrastructure, and changing external services.

Hybrid Optimize was my first opportunity to own that process completely.

AI-assisted case-study methodology

I used OpenAI Codex to help reconstruct the implementation history from reviewed evidence, compare it with the public documentation and research paper, challenge the scope of my claims, organize the narrative, and generate an initial draft of this article.

The original product opportunity, architecture, implementation, benchmarks, and engineering decisions described here came from my work at Strangeworks. The prose and diagrams were developed later with AI assistance under my direction and fact-checked against my records. No private Strangeworks source code, credentials, customer data, or internal schemas have been reproduced.