Agentic Aggregation: Treating Consensus as an Investigation Rather Than a Vote

The pursuit of scaling test-time compute has fundamentally altered how we conceptualize intelligence in large language models. Where once we focused exclusively on training larger models, recent work by Wei et al., Snell et al., and others has demonstrated that allowing models to think longer, sample multiple solutions, or verify their own work can yield dramatic improvements in reasoning capabilities. Yet this scaling has largely been validated on constrained domains such as mathematical proof generation or code synthesis, where solutions are verifiable and trajectories are relatively short. The paper "Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Tasks" by Lee, Yen, Ye, and Chen from Princeton University confronts a substantially more difficult challenge: how do we aggregate multiple independent attempts at complex, multi-turn tasks like deep research or web navigation, where evidence is sparse, distributed across hundreds of steps, and interleaved with tool calls?

The central contribution of this work is AggAgent, a system that reframes aggregation not as a summarization or voting problem, but as an agentic task in its own right. Rather than treating parallel trajectories as static texts to be compressed, AggAgent treats them as an interactive environment, navigating through specific trajectories on demand to synthesize a final answer. This shift from passive consolidation to active investigation represents a meaningful evolution in how we approach test-time scaling for open-ended agentic systems.

The Architecture of Selective Synthesis

The technical innovation of AggAgent lies in its recognition that full trajectory concatenation is infeasible and summarization is lossy. Long-horizon agentic tasks frequently generate hundreds of thousands of tokens per trajectory across multiple turns of tool use and observation. Loading eight such trajectories into a context window exceeds the capacity of even the most generous current models, while compressing them into summaries risks discarding the specific evidentiary snippets that distinguish a correct solution from a convincing hallucination.

To circumvent this, the authors equip AggAgent with three lightweight tools that operate exclusively over the array of completed trajectories: get_solution (retrieving final answers from specific runs), search_trajectory (keyword search within a single trajectory), and get_segment (random access to specific step ranges). These tools confer several crucial properties. First, they enable full fidelity access to source material without requiring the model to attend to all trajectories simultaneously. Second, they impose zero external cost; unlike the rollouts themselves, which may invoke expensive web search APIs or code execution environments, these aggregation tools operate entirely over cached trajectory data. Third, they bound the aggregation cost to approximately that of a single additional agentic rollout, adding only 5.7% overhead compared to the 41% incurred by Summary Aggregation methods that process each trajectory independently.

This tool-based approach effectively decouples the information retrieval budget from the generation budget. The aggregator can perform targeted investigations, searching for specific entities across trajectories or retrieving particular reasoning segments when discrepancies emerge between candidate solutions. This stands in contrast to Solution Aggregation, which discards all intermediate reasoning, or Majority Voting, which relies on superficial answer frequency and fails entirely on open-ended tasks with non-identical correct answers.

Empirical Performance and the Synthesis Advantage

The empirical evaluation spans six benchmarks across three distinct model families: GLM-4.7, Qwen3.5, and MiniMax-M2.5. The selection includes BrowseComp and Humanity's Last Exam (HLE), alongside other deep research and agentic search tasks. This diversity is significant; it demonstrates that the aggregation strategy generalizes beyond specific base model architectures or particular tool configurations.

The results are striking. AggAgent achieves up to 5.3% absolute improvement over existing aggregation methods on average, with gains of 10.3% on deep research tasks specifically. These margins are substantial in the context of agentic benchmarks, where performance often clusters tightly. More surprisingly, the authors report that AggAgent surpasses Pass@8, the theoretical upper bound representing the probability that at least one of the eight parallel trajectories contains a correct answer. This implies that AggAgent is not merely selecting the best trajectory, but actively synthesizing correct information from multiple partially correct trajectories to construct solutions superior to any individual constituent.

The cost-accuracy Pareto frontier reveals the practical significance of this approach. While Summary Aggregation offers competitive accuracy, its computational cost scales linearly with the number of trajectories. AggAgent achieves comparable or superior accuracy while maintaining costs comparable to simple heuristics like Best-of-N. This efficiency stems from selective attention; the agent inspects only the trajectory segments relevant to resolving specific uncertainties, rather than processing all available text.

Beyond Voting: Meta-Reasoning and Its Limitations

What makes AggAgent conceptually distinct is its alignment with the broader trend toward meta-cognitive or System 2 reasoning in artificial intelligence. Traditional ensemble methods treat individual trajectories as black boxes emitting answers; AggAgent treats them as evidentiary records requiring cross-examination. This mirrors human scientific practice, where multiple studies are not simply voted upon but carefully compared, contrasted, and synthesized to resolve contradictions.

However, this approach introduces new vulnerabilities that warrant careful consideration. The efficacy of AggAgent depends critically on the design of its inspection tools. The current implementation relies on keyword search for trajectory navigation; while efficient, this may miss semantic matches where relevant information is expressed using different terminology than the query. A trajectory containing the answer "Eric Adams" might not surface if the aggregator searches for "Mayor of New York" and the trajectory uses pronouns or alternative references. More sophisticated retrieval mechanisms, such as semantic search or learned trajectory embeddings, might improve recall but would introduce additional computational overhead, potentially breaking the cost efficiency that makes this method attractive.

Furthermore, AggAgent inherits the classic verification bottleneck: the aggregator itself can hallucinate or misinterpret the trajectories it inspects. The paper notes that aggregation cost remains bounded by a single agentic rollout, but this also means the aggregator operates under the same cognitive constraints as the base agents. If multiple trajectories contain mutually consistent but incorrect reasoning, the aggregator may compound these errors rather than resolve them. The method assumes that diversity among parallel rollouts produces complementary correct information rather than systematic shared misconceptions, an assumption that may fail on adversarially constructed or highly biased tasks.

There is also an open question regarding hierarchical scaling. The current formulation aggregates K parallel trajectories into a single output. For extremely complex tasks, one might imagine recursive aggregation, where groups of trajectories are first condensed into intermediate reports, which are then aggregated themselves. Whether such hierarchical approaches preserve the accuracy gains while maintaining cost bounds remains unexplored.

Conclusion: The Future of Parallel Cognition

The work on AggAgent suggests a fundamental reorientation in how we allocate computational resources at inference time. The field has focused intensely on generating more samples, but this research demonstrates that how we inspect those samples matters as much as how many we generate. The trajectory is no longer merely a means to an answer; it is a structured record of exploration that contains valuable information about dead ends, alternative hypotheses, and evidentiary provenance.

Looking forward, several research directions emerge. First, the toolset for trajectory inspection could be expanded to include cross-trajectory diffing, temporal alignment of parallel search processes, or confidence estimation based on tool call redundancy. Second, the principles of agentic aggregation might be applied to other forms of test-time scaling, such as aggregating across different reasoning chains in mathematical domains or combining outputs from specialist subagents in modular systems.

Ultimately, AggAgent represents a step toward agents that not only act in parallel but reason about their own distributed cognition. As we push toward more autonomous systems capable of deep research and extended problem-solving, the ability to synthesize diverse attempts into coherent, verified conclusions will prove essential. The aggregation problem, once treated as a post-processing step, reveals itself to be a complex reasoning task worthy of agentic treatment in its own right.