Sometimes you hit a task with two critical demands β for example, producing an output while guaranteeing zero errors β and no single model satisfies both constraints. The easy temptation is to pick the best available model and accept a compromise on quality. A better approach is to combine two models in a pipeline.
The basic idea
The pattern is simple. A primary model produces the initial output, and a verifier model evaluates it to catch errors the primary missed. Each model handles the task it is best suited for. This division of labour creates a composite system that meets strict requirements, even when no individual model can.
The key rule: the verifier must genuinely be capable
Ignoring the decisive rule of this pattern breaks the implementation: the verifier must be genuinely capable on the exact axis it checks. If the verifier is weak at detecting errors, it provides nothing more than a worthless stamp of approval.
A second, less obvious rule applies: the primary must not be so weak that the verifier cannot distinguish subtle errors from total failure. A strong verifier cannot salvage a thoroughly broken output; it can only catch edge-case errors in an otherwise sound one. The primary should sit only one band below the demand, not two. Combining a capable primary with a strict verifier yields a critical-level result; combining a weak primary with that same verifier fails.
Three shapes of pipeline
This pattern has three common implementations, each with different economics:
- Sequential: The primary produces, then the verifier approves or rejects. All traffic passes through both, meaning cost and latency equal the sum of the two models. This is the simplest and most reliable architecture.
- Tiered fallback: The verifier runs only on a flagged subset of outputs, such as low-confidence or suspicious cases. Cost is significantly lower because the verifier processes only a fraction of the traffic.
- Parallel arbitration: Both models run independently, and a deterministic rule arbitrates between their outputs. This incurs the highest cost and maximum latency, but is necessary when strictly independent evaluations are required.
Choosing between these three is an engineering trade-off governed by task sensitivity, latency budgets, and cost constraints.
When not to reach for this pattern
This pattern is not free. Two models introduce greater latency, higher costs, and more operational complexity. It is justified only when no single model can meet the requirements. If one model can handle the task, use it β it will be simpler, cheaper, and have fewer failure points. Reserve this pipeline for scenarios where the pool of single-model candidates is genuinely empty.
An honest caveat
Combining a primary and a verifier does not provide an absolute guarantee; it only sharply reduces the probability of error. You must still measure in practice whether the paired system reaches your target quality threshold. The pattern is a useful architectural tool, but it does not replace rigorous evaluation.