Evaluation of major AI powered answer engines

On November 30, 2025, we evaluated answers from the top 5 AI-powered answer engines: ChatGPT, Perplexity, Google AI Mode, Grok, and Ask Brave ― using state-of-the-art Large Language Models (LLMs) as judges. The evaluation covers 1500 queries randomly sampled from real-world usage.

We scraped each search engine, collecting answers as presented to end users, using BrightData's web scraping services (except for Ask Brave, which we accessed directly). We chose scraping over APIs to capture the experience as a typical user would encounter it — specifically, the free tier without login, representing the most common entry point for new users.

For the evaluation, we used the prompt in Annex II with Anthropic's Claude Opus 4.5 and Claude Sonnet 4.5 as judges. To maximize consistency, we conducted all possible pairwise comparisons rather than evaluating all engines simultaneously, as pairwise judgments yield more reliable assessments from LLM judges. Each pair was evaluated twice with reversed ordering to eliminate position bias. Detailed results for every comparison (e.g., Ask Brave vs. ChatGPT, ChatGPT vs. Google AI Mode) appear in Annex I, including both 5-point Likert scale ratings and win/draw/lose ratios.

Grok scores best with Ask Brave following closely despite using a lower performance open-weights model (Qwen3). The key difference in these tests was Ask Brave’s use of the higher quality grounding context endpoint—the same API we're releasing today. Systems with limited search index access trail significantly, demonstrating that context quality matters more than model capability.

Google AI Mode is 3rd, with a noticeable gap ― despite access to a full-scale search index and frontier models. We speculate, however, that this may not represent Google's best effort ― performance could be throttled to manage costs at massive scale.

The last two are ChatGPT and Perplexity which have limited/partial access to a search index. We speculate that Perplexity may not use OpenAI models outside its paid tiers, which could explain its weaker performance if it relies on a less capable open-weight model. This analysis is necessarily speculative ― little is publicly known about competitors' architectures.

The one system we can describe with certainty is Ask Brave: it uses exclusively the Brave LLM context API for retrieval and Qwen 3 for inference, a combination that runs cost-effectively at scale. As of this writing, Ask Brave serves approximately 600K queries per day despite being accessible only via an explicit tab click ― it is not the default in any user flow.

Direct Comparison:

BOTH EQUAL      │███████████████████████│ 32.00%
AI Mode         │████████████████████████████████████████│ 54.53%
Perplexity      │█████████│ 13.47%

Average Grades (5 point Likert Scale):

AI Mode         │█████████████████████████████████████│ 4.64
Perplexity      │████████████████████████████████│ 4.09

Direct Comparison:

BOTH EQUAL      │█████████████████████████████│ 38.27%
Ask Brave       │████████████████████████████████████████│ 51.80%
AI Mode         │███████│ 9.93%

Average Grades (5 point Likert Scale):

Ask Brave       │█████████████████████████████████████│ 4.72
AI Mode         │██████████████████████████████████│ 4.27

Direct Comparison:

BOTH EQUAL      │██████████████████████████│ 35.13%
Ask Brave       │████████████████████████████████████████│ 53.33%
ChatGPT         │████████│ 11.53%

Average Grades (5 point Likert Scale):

Ask Brave       │█████████████████████████████████████│ 4.72
ChatGPT         │█████████████████████████████████│ 4.22

Direct Comparison:

BOTH EQUAL      │████████████████████████████████████████│ 47.07%
Ask Brave       │██████████████│ 16.53%
Grok            │██████████████████████████████│ 36.40%

Average Grades (5 point Likert Scale):

Ask Brave       │███████████████████████████████████│ 4.42
Grok            │█████████████████████████████████████│ 4.64

Direct Comparison:

BOTH EQUAL      │██████████│ 19.38%
Ask Brave       │████████████████████████████████████████│ 75.19%
Perplexity      │██│ 5.43%

Average Grades (5 point Likert Scale):

Ask Brave       │██████████████████████████████████████│ 4.78
Perplexity      │███████████████████████████████│ 3.95

Direct Comparison:

BOTH EQUAL      │████████████████████████████████████████│ 38.73%
ChatGPT         │███████████████████████████│ 26.87%
AI Mode         │███████████████████████████████████│ 34.40%

Average Grades (5 point Likert Scale):

ChatGPT         │███████████████████████████████████│ 4.41
AI Mode         │████████████████████████████████████│ 4.52

Direct Comparison:

BOTH EQUAL      │█████████████████│ 27.73%
ChatGPT         │█████│ 9.07%
Grok            │████████████████████████████████████████│ 63.20%

Average Grades (5 point Likert Scale):

ChatGPT         │████████████████████████████████│ 4.08
Grok            │█████████████████████████████████████│ 4.72

Direct Comparison:

BOTH EQUAL      │████████████████████████████│ 34.07%
ChatGPT         │████████████████████████████████████████│ 48.00%
Perplexity      │██████████████│ 17.93%

Average Grades (5 point Likert Scale):

ChatGPT         │████████████████████████████████████│ 4.58
Perplexity      │█████████████████████████████████│ 4.21

Direct Comparison:

BOTH EQUAL      │███████████████████│ 30.07%
Grok            │████████████████████████████████████████│ 60.53%
AI Mode         │██████│ 9.40%

Average Grades (5 point Likert Scale):

Grok            │█████████████████████████████████████│ 4.70
AI Mode         │█████████████████████████████████│ 4.14

Direct Comparison:

BOTH EQUAL      │███████│ 15.47%
Grok            │████████████████████████████████████████│ 79.33%
Perplexity      │██│ 5.20%

Average Grades (5 point Likert Scale):

Grok            │██████████████████████████████████████│ 4.76
Perplexity      │██████████████████████████████│ 3.80

Below is the prompt template used for each pairwise comparison. The template variables are:

• query (or "prompt"): the query under evaluation
• answer1: the response from the first AI search engine
• answer2: the response from the second AI search engine

As noted earlier, each pair was evaluated twice with answer order reversed to control for positional bias (i.e., a tendency to favor whichever option appears second).

Notes on answer formatting:

• We evaluate the Markdown representation of each answer, including section headings and other structural elements.
• Ask Brave uses a custom <augment> tag as an extension to Markdown, enabling inline UI elements such as embedded web results. These tags are treated as part of the answer, analogous to tables, inline images, or other Markdown enrichments.
• Inline references (e.g., citation markers) are excluded from evaluation, as they are not consistently available across all engines and are often rendered as UI-only elements rather than answer content.

Notes on prompt design:

• The evaluation prompt is intentionally minimal. By avoiding explicit criteria, we reduce the risk of biasing the assessment toward aspects we deem important and instead allow the LLM judges to reason freely.

You are a human reading two answers to the same query. Your task is to grade each answer on a scale from 1 to 5.

Grading scale:
- 1: Incorrect or harmful
- 2: Mostly incorrect or missing critical information
- 3: Partially correct with significant gaps
- 4: Mostly correct with minor issues
- 5: Correct and addresses the query well

Answers are in Markdown. Evaluate them as they would appear when rendered, not as raw text.

Some answers contain `<augment>...</augment>` tags. These are placeholders for additional content that would be displayed to the user.

<references guidance>
IMPORTANT: Do NOT evaluate or consider inline references, citations, or source links when assessing the answers. References (such as [1], [2], footnotes, or hyperlinks to sources) should have NO impact on your evaluation. Focus only on the actual content and substance of the answer itself.
</references guidance>

<factuality guidance>
IMPORTANT: Do NOT assess the factual accuracy of the information provided. Assume all information in the answer is correct but detect when answers contradict each other.
</factuality guidance>

<QUERY>{query}</QUERY>
<USER SEARCH COUNTRY>us</USER SEARCH COUNTRY>

<ANSWER1>{answer1}</ANSWER1>

<ANSWER2>{answer2}</ANSWER2>

Think step-by-step and provide your reasoning between <REASONING> and </REASONING> tags, then give your final verdict.

Expected answer structure:

<REASONING>Explain your thought process here, evaluating each answer.</REASONING>
<EXPLANATION>Brief summary of your evaluation</EXPLANATION>
<ANSWER1_GRADE>integer from 1 to 5</ANSWER1_GRADE>
<ANSWER2_GRADE>integer from 1 to 5</ANSWER2_GRADE>