Research and Synthesis at Scale

The naive move when you want a researched answer is to hand one agent the question, point it at the web or a folder of documents, and ask for the report. It comes back fluent, structured, footnoted, and you cannot tell whether it is right. That is the trap. The work reads as authoritative precisely because the model is good at the part that does not matter, the prose, while the failure hides in the part that does, whether the claims are grounded in anything real. A growth team that swapped its analytics stack for one Claude agent named the result exactly: "garbage in, compelling-looking garbage out, now with AI confidence." Scale does not improve this; it makes a wrong answer look more finished.

The structural reason is that one agent reading many sources is doing two incompatible jobs in one context window, and both degrade as the window fills. The first job is extraction, reading each source closely and pulling what it says. The second is synthesis, holding everything at once and reconciling it into one account. Pile ten sources into a single window and the early ones drift out of working attention while the model is still reading the late ones, so by the time it writes, it is synthesizing from a half-remembered blur rather than from the sources in front of it. This is the same long-input decay that bites long-form generation. A builder who shipped fifty thousand articles found the model "loses recency, anything in the first 30% of the system prompt drifts out of working attention," and tested it across three frontier models that all drifted past the same point. It is not a model-size problem. It is what happens when one window carries the whole job.

The fix is to split the two jobs the way the work actually divides. Fan the extraction out: one sub-agent per source, each reading its source in a clean window and returning a tight structured summary, never the raw text. Then a dedicated synthesis step takes those summaries, not the originals, and reconciles them into the answer. The synthesis agent never reads forty documents. It reads forty digests, which fit. This is the move that turns "more sources" from a liability into a strength, because reading scales horizontally across parallel windows while the thing that does not scale, reconciliation, runs once over compressed inputs.

The pipeline has a recognizable shape: pose the question precisely, fan out per-source extraction, enrich the gaps the first pass missed, synthesize once, then verify. This very Handbook was built that way. Twenty-one extraction agents each read one research surface and returned a structured digest; a later synthesis pass reconciled those digests into the source map a module author reads, not by re-reading twenty-one raw briefs but by reading their summaries. The document you would author against is itself the output of this pipeline.

But fan-out and synthesis only get you a well-organized answer. They do nothing for whether the answer is true, and a confidently organized falsehood is worse than an obvious one because it survives review. This is where the gate earns its place. The drafting failure that real pipelines guard against is precise: the model will invent a plausible-sounding number to fill a gap, and "those numbers will survive editorial review unless an editor opens every link." The countermeasure is a hard constraint stated upfront, that every statistic must trace to a source in the supplied set, paired with a verification pass whose only job is to walk each claim back to its source and flag the ones that have none. That pass is not editing for flow. It is a distinct step asking one question per claim: where in the supplied sources does this come from?

The trace-every-claim gate is the load-bearing part, and it is also where the discipline most often quietly fails. The first failure is the hallucinated figure that reads like a finding. A model will narrate "conversion rose from 3.2% to 4.8%" with the same cadence it uses for real numbers, and if your gate only checks claims that look uncertain, the confident ones walk straight through. The gate has to be indiscriminate, checking every number rather than the suspicious-looking ones, because the dangerous fabrications are the ones that sound the most settled. This Handbook's own pipeline caught a fabricated chart midpoint in a finished module during exactly this kind of pass; it had survived the draft because it looked like every other grounded number around it.

The second failure is synthesis that smooths over disagreement instead of surfacing it. When two sources conflict, the model's instinct is to blend them into one untroubled sentence, because agreeable prose is what it was trained to produce — and that blend quietly omits the conflict, which is often the most useful signal in the pass. Practitioners have named the reflex bluntly: LLMs "try to agree with everyone," and the same instinct that flattens a content draft flattens a research synthesis. Operator-grade synthesis does the opposite. It names the disagreement, attributes each side, and says which is better supported, because the conflict is often the most useful thing in the whole pass. If your synthesis never reports that two sources disagreed, suspect it is hiding the disagreement, not that the sources happened to align.

The third failure is fan-out that outruns synthesis. It is cheap to spawn forty extractors and expensive to reconcile forty digests well, so the temptation is to scale the reading and assume the synthesis will keep up. It will not. A single synthesis context has a real ceiling. Past roughly a handful of dense batches it starts compressing rather than reconciling, summarizing the summaries and losing the cross-source contradictions that were the point. The constraint to respect is that fan-out should produce only as much as one synthesis pass can actually hold and reconcile. If the synthesis step is drowning, the answer is fewer, denser extractions or a tiered synthesis, not a bigger pile.

And the whole pipeline is the wrong tool when the question has one source or no real sources. If the answer lives in a single document, fanning out is theatre, because one careful read beats five parallel partial ones. If the question is genuinely open-ended judgment with nothing to ground against, no amount of extraction produces a grounded claim, because there is nothing to trace to. The pipeline earns its cost only when there are many real sources and the risk is that one agent cannot hold them all honestly at once.