Perplexity AI Unveils WANDR: A Groundbreaking Benchmark for Wide and Deep AI Research Agents

The Evolving Landscape of AI Research Agents

The bigger takeaway is simple: In today’s fast-paced digital world, AI-powered research agents are increasingly taking on complex knowledge work. From competitive mapping and due diligence to comprehensive literature reviews, these agents are becoming indispensable tools for various teams. However, a significant challenge has persisted in their evaluation: most existing benchmarks primarily test an agent’s ability to provide a single, accurate answer, rather than their capacity to build large, evidence-backed collections of information.

Meanwhile, Addressing this critical gap, Perplexity AI has introduced WANDR (Wide ANd Deep Research), an innovative open benchmark designed to rigorously assess research agents that must not only search broadly but also delve deeply to substantiate every claim with verifiable evidence.

What is WANDR? Defining “Wide” and “Deep” Research

WANDR stands as an open benchmark and evaluation harness, meticulously crafted around 500 realistic and challenging data-collection tasks pertinent to professional knowledge work. It serves as a complementary sibling to Perplexity’s DRACO benchmark, which focuses on deep research by evaluating whether an agent can produce accurate, complete, and objective long-form reports.

In practical terms, WANDR, in contrast, specifically asks whether an agent can construct a substantial collection of information, each piece supported by concrete evidence. At its core, WANDR simultaneously tests two crucial demands:

• Wide: This refers to the agent’s ability to discover a large, often open-ended set of qualifying entities relevant to a given task. It’s about breadth of discovery.
• Deep: This involves thoroughly investigating each identified entity to ensure every claim made about it is fully supported by verifiable evidence. It’s about depth of validation.

The combination of these two demands fundamentally alters the problem for AI agents. It signifies that a few compelling examples are insufficient, and a well-articulated narrative built on incomplete research will simply not pass muster.

The Hierarchical Approach to Task Definition

For example, To accurately capture the complexity of wide and deep research, WANDR employs a composable qualification key hierarchy. This flexible structure allows tasks to represent everything from flat lists to intricate nested searches or matrices.

For instance, a task might request data structured as company(n) -> employee(m) -> url(k), meaning ‘n’ qualifying companies, each with ‘m’ employees, and ‘k’ supporting web pages for each employee entry. Every complete path through this hierarchy is independently validated.

A concrete example is the ceo_cfo_appointments task. This requires agents to identify at least 70 US-based companies with a CEO or CFO appointment announced within a specific timeframe (e.g., March 1 to April 30, 2026). For each company, the agent must provide:

• One authoritative appointment page URL.
• One listing-authority page URL per company.

That said, Collectively, this single task demands the collection of 140 source-backed records, demonstrating the benchmark’s focus on scale and evidentiary support.

Building Realistic and Scalable Tasks

WANDR’s tasks are not synthetically generated; they are meticulously built from real, de-identified patterns observed in production environments. A semi-automated pipeline, involving stages like seeding, authoring, admission, and curation, transforms these real-world patterns into robust tasks. This pipeline utilizes an interleaved author-critic loop combined with mechanical linting to ensure quality and relevance.

Interestingly, The scale of WANDR is impressive: across all 500 tasks, the benchmark collectively calls for an astounding 170,495 source-backed records. The median task requires agents to identify 50 members and collect 245 records overall. Tasks are further categorized into lower, middle, and higher difficulty levels, with difficulty determined by the per-record work involved, not just the sheer volume.

How WANDR Grades Agent Performance

Unlike benchmarks that rely on fixed answer keys, WANDR employs a sophisticated, reference-free grading mechanism. Each submitted claim is individually verified against its cited evidence.

The grader dynamically re-fetches the provided URL during evaluation to confirm the page’s usability and scope. It then meticulously checks whether the cited excerpts genuinely appear on the page and fully support every requirement of the claim.

However, These binary record verdicts are then rolled up through the hierarchy to produce overall scores:

• Precision: Measures the quality of the information submitted by the system.
• Recall: Assesses the quality-adjusted completion, penalizing for any missing information.
• Soft Scores: Provide partial credit for incomplete members or subtrees.
• Hard Scores: Only count members whose entire subtree of information is perfectly correct and complete.

Initial Benchmark Results and Key Findings

Perplexity AI evaluated six production systems, including its own Search as Code (SaC) system, across all 500 WANDR tasks. While Perplexity’s SaC system currently leads the leaderboard, achieving a 0.363 soft F1 and 0.133 hard F1 score, no system has yet come close to fully solving the benchmark.

Meanwhile, The cost associated with running these tasks also varied dramatically, spanning more than four orders of magnitude, from a mere $0.03 per task to an exorbitant $324.83 per task depending on the settings.

Beyond the raw scores, four critical findings emerged:

1. Partial Progress is Common, Complete Coverage is Not: Every system demonstrated soft recall scores consistently below soft precision, indicating a struggle to achieve full completion even when individual pieces of information were correct.
2. Scale Compounds the Problem: Deeper hierarchical tasks significantly amplified the difficulty, as each branch introduced a new potential point of failure for the agents.
3. Discovery is the Primary Bottleneck: The initial discovery phase proved to be the most significant structural bottleneck. Under-delivery of initial qualifying entities, rather than issues with merging duplicates, accounted for the majority of missing volume.
4. Evidence Extraction is the Hardest Part: While finding a usable web page was generally straightforward, transforming that page into complete and verifiable evidence presented a major hurdle. For Perplexity’s own system, 41.4% of pages missed a substantive requirement, and a staggering 57.5% of excerpts failed to fully support the associated claim.

In practical terms, Notably, the programmatic nature of Perplexity’s Search as Code system appears well-suited to this task shape, allowing agents to express complex logic for retrieval, filtering, fan-out, joins, deduplication, and stopping as executable programs.

Real-World Applications of WANDR

The types of tasks within WANDR directly mirror real-world applications where teams already seek to automate knowledge work. For instance:

• A market analyst needs to identify every qualifying competitor, complete with supporting evidence for each.
• A due diligence team requires detailed information on dozens of companies, including their ownership structures, key executives, and financing details.
• Talent sourcing professionals need to discover numerous candidates, each with supporting profile pages and verified credentials.

For example, WANDR precisely tests these wide-and-deep collection patterns at a professional scale. Furthermore, its per-record grading system allows engineers to localize failures with precision, isolating issues to discovery, enrichment, or evidence extraction stages, thereby aiding targeted improvements.

Expert Perspective

From an industry angle, the clearest signal around AI Research Agents Benchmark is how it may influence wandr. The story reads less like a one-day spike and more like a marker of broader movement.

The next phase will depend on how quickly teams, regulators, or customers react. In practice, that gives AI Research Agents Benchmark room to reshape expectations across research over the near term.

For readers focused on practical impact, the best next step is to watch what changes around agents once attention turns into execution.

Frequently Asked Questions

Why does AI Research Agents Benchmark matter right now?

The Evolving Landscape of AI Research AgentsThe bigger takeaway is simple: In today’s fast-paced digital world, AI-powered research agents are increasingly taking on complex knowledge work.

What broader change could AI Research Agents Benchmark signal?

From competitive mapping and due diligence to comprehensive literature reviews, these agents are becoming indispensable tools for various teams.

What should the market watch next around AI Research Agents Benchmark?

However, a significant challenge has persisted in their evaluation: most existing benchmarks primarily test an agent’s ability to provide a single, accurate answer, rather than their capacity to build large, evidence-backed collections of information.Meanwhile, Addressing this critical gap, Perplexity AI has introduced WANDR (Wide ANd Deep Research), an innovative open benchmark designed to rigorously assess research agents that must not only search broadly but also delve deeply to substantiate every claim with verifiable evidence.What is WANDR?

Key Takeaways

• WANDR is an open benchmark featuring 500 evidence-heavy, wide-and-deep data collection tasks.
• Tasks are defined using a flexible qualification key hierarchy, with validation occurring path-by-path.
• Grading is reference-free; the system dynamically re-fetches and verifies cited evidence.
• Perplexity’s Search as Code system currently leads with 0.363 soft F1 and 0.133 hard F1, but no system has yet mastered the benchmark.
• The biggest challenges for agents remain initial discovery and the ability to extract complete and verifiable evidence from sources.

WANDR represents a significant step forward in evaluating the capabilities of AI research agents, pushing them beyond simple question-answering towards truly comprehensive, evidence-based knowledge collection.

Source: https://www.marktechpost.com/2026/07/19/perplexity-ai-releases-wandr-an-open-benchmark-evaluating-research-agents-that-must-search-wide-and-deep/