Quick answer: Which AI search tool performed better?
In this small, unblinded pilot, ChatGPT Search received higher preliminary scores. That is not proof it is universally more accurate.
The published provisional worksheet scores Google AI Mode at 92.57/100 and ChatGPT Search at 100/100. Six questions were marked ties and eight received a higher first-pass ChatGPT score. Yet the same ChatGPT assistant that participated in the comparison also judged the answers. We therefore treat these as exploratory editorial notes, not a validated winner.
The useful finding is more specific: careful source checking uncovered a potentially consequential pricing-tier error, while science, legal and SEO prompts revealed how citation quality and precise wording affect the usefulness of an apparently polished AI answer.
If you're deciding which tool to trust, use this audit as a guide to where to verify, not as evidence that one tool wins every research task.
All 14 questions: Google AI Mode vs ChatGPT Search
Every question was supplied in identical English wording to both products. Numbers below are the original, unblinded editorial assessments. Zero additional reruns and zero independent human reviews are claimed.
| Test | Research task | ChatGPT | Skill examined | |
|---|---|---|---|---|
| T01 | Anthropic vault particles | 100 | 100 | Fact retrieval |
| T02 | Anthropic FlashProve | 100 | 100 | Recent technical news |
| T03 | Google AI search link changes | 100 | 100 | Official product updates |
| T04 | EU AI Act GPAI enforcement | 96 | 100 | Regulatory precision |
| T05 | Attention Is All You Need | 100 | 100 | Bibliographic retrieval |
| T06 | JWST life claim on K2-18 b | 78 | 100 | Contested science |
| T07 | Hubble vs Webb | 92 | 100 | Scientific comparison |
| T08 | 2026 Nobel Prize in Chemistry | 98 | 100 | Exact official wording |
| T09 | 95% confidence interval | 100 | 100 | Statistical reasoning |
| T10 | Agent pass^k versus pass@k | 97 | 100 | Quantitative reasoning |
| T11 | GPT-6 Astra vs Claude pricing | 58 | 100 | Cost-sensitive retrieval |
| T12 | Python 3.14 free-threading | 100 | 100 | Developer documentation |
| T13 | Does llms.txt help Google AI Mode? | 94 | 100 | SEO misinformation |
| T14 | Search crawlers vs training bots | 83 | 100 | Publisher controls |
Both products correctly answered many factual or mathematical tasks. The apparent ceiling effect—particularly ChatGPT's 14 full marks—makes the independent-review caveat unusually important. See the full prompt-by-prompt appendix and source score worksheet (CSV).
The most consequential example: a $270 quote for a $45 workload
Test T11 asked both systems to compare standard, uncached, non-batch API token rates for GPT-6 Astra and Claude Sonnet 4.6, then price a fixed workload: 2 million input tokens and 500,000 output tokens. This was not an open-ended request for the fastest service tier.
Applied $60 input / $300 output per million for GPT-6 Astra, associated with a different, higher-priced processing tier.
At $10 input / $50 output per million, calculated as (2 × $10) + (0.5 × $50).
| Model and tier | Input /1M | Output /1M | Example total |
|---|---|---|---|
| GPT-6 Astra — Standard | $10 | $50 | $45.00 |
| Claude Sonnet 4.6 — Standard | $3 | $15 | $13.50 |
| GPT-6 Astra — alternative premium tier cited in first Google answer | $60 | $300 | $270.00 |
Both answers did arithmetic correctly for their chosen rates. The failure was in selecting the correct tier from an official pricing page—precisely the kind of subtle source-use error that a visible citation may not expose. Prices are a dated November 5, 2026 snapshot, not an evergreen quote. Long-context surcharges, tokenization, tools and other terms may change the invoice. Check OpenAI's API pricing and Anthropic's pricing before purchasing.
Science fact-checking: confidence is not evidence
Some prompts were designed to separate reported observations from stronger claims. T06 asked whether NASA's James Webb Space Telescope had confirmed alien life on K2-18 b. Both products rejected the viral claim. The differences appeared in how cautiously they handled tentative dimethyl sulfide signals, carbon-dioxide uncertainty and peer-reviewed reanalyses.
The first editorial pass favored an answer that distinguished the underlying JWST spectra, possible atmospheric molecules, alternative non-biological explanations, and the debate between scientific teams. The correct user takeaway remains straightforward: Webb has not confirmed life on K2-18 b. See NASA's K2-18 b announcement and Stevenson et al.'s peer-reviewed reanalysis.
T07 compared Hubble and Webb using NASA pages. T08 required the exact wording of the 2026 Chemistry Nobel citation. T04 tested whether the answer distinguished EU AI Act applicability from enforcement and the correct institutional remit. These are not merely fact-recall tasks: an answer can be broadly correct while overstating certainty, quoting a paraphrase as official wording, or misidentifying responsibility.
In contrast, the easier source-finding prompts T01–T05 were near the top of the scoring range for both systems. That ceiling effect is a reason to resist treating every score difference as equally meaningful.
AI Search SEO: what the crawler tests revealed
Two late-stage prompts focused on questions a working publisher actually faces. T13 asked whether llms.txt was mandatory for Google AI Mode or improved Google ranking. Both products rejected the myth. Google's published guidance says no separate AI-specific text file is required for Google Search. A more helpful answer also distinguishes crawl eligibility, snippet eligibility and participation controls. Read Google's official generative-AI SEO guide.
T14 asked for a publisher's robots.txt strategy to preserve discovery in Google AI Mode and ChatGPT Search while opting out of relevant training uses. Both proposed the central four-token configuration:
User-agent: Googlebot
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: Google-Extended
Disallow: /
User-agent: GPTBot
Disallow: /But implementation details matter. Google-Extended is a product-use control, not an independently identifiable HTTP crawler; GPTBot and OAI-SearchBot have different purposes; and robots directives are not a guarantee of ranking, citations, or deletion of previously collected content. Check Google's crawler controls and OpenAI's bot documentation. This sample snippet must be adapted to any existing rules and the site's other visibility settings.
Our test method: one paired answer per question
Research ownership. SXF Research designed the study, selected the 14 research questions, and documented and published the findings. Google AI Mode and ChatGPT Search were the platforms evaluated; neither Google nor OpenAI conducted or authored this SXF Research study.
Design. On November 5, 2026, one tester submitted 14 English prompts to Google AI Mode and ChatGPT Search, recording one answer from each for every task. Prompts targeted news, science, legal interpretation, quantitative reasoning, software documentation, API pricing and search-crawler controls. The 14 prompts were deliberately selected, not randomly sampled from an external query distribution.
Scoring. A ChatGPT assistant compared the pasted responses against primary references and gave each response an editorial score out of 100: factual accuracy (40%), source support (30%), currency/relevance (20%) and clarity/usefulness (10%). The scores were assigned on a 0–5 scale per dimension. This was not blinded, independent or externally validated.
Preliminary evaluator conflict. ChatGPT assisted in scoring both Google AI Mode and ChatGPT Search, including its own responses. No independent human review, blinded evaluation or independent validation was performed.
What we did not collect. The precise individual test-run timestamps, location/device configuration, reproducibility settings, full original share links and verified verbatim export of all 28 responses were not preserved as complete independent records. Some response texts in the spreadsheet were subsequently edited for clarity or abbreviated. We explicitly disclose this in the appendix and do not pass those extracts off as raw output.
No statistical winner. One answer per product per prompt, hand-selected subjects, no repeated runs and a judge who is also a contestant cannot establish a general model ranking or a confidence interval for real-world quality. View the detailed protocol and open files.
Limitations: the reason this is an exploratory audit, not a benchmark
- Evaluator conflict: ChatGPT assigned itself 100/100 in all 14 cases. Those scores are provisional and should be reviewed blind by independent assessors.
- Small purposive sample: 14 chosen tasks can illustrate failure modes, not represent all user searches.
- One run only: No repeatability, within-tool variance or error-frequency estimates were measured.
- Archival fidelity: Original responses were supplied in this conversation, but the downloadable spreadsheet versions include normalized or abbreviated text and must not be cited as a complete verbatim archive.
- Changing facts: Price tables, AI Search policies and scientific interpretations may change after the observation date.
These constraints do not erase the examples; they define what the evidence can support. A future independent blinded study with multiple runs could confirm, weaken or reverse the preliminary ordering.
A verification checklist for AI Search users
- Identify the exact decision. Does a wrong number cost money, change a legal interpretation or affect production infrastructure?
- Open the cited page. Verify that the referenced section actually supports the specific claim—not merely the topic.
- Match the context. Check plan/tier, effective date, currency, jurisdiction, model version, and whether the setting is default or optional.
- Seek conflicting primary analysis. For contested science, find independent peer-reviewed reanalyses rather than relying on a single positive signal.
- Recalculate high-impact totals. Independent arithmetic is quick; selecting the correct input rates may be harder.
- Separate eligibility from guarantees. Allowing a crawler does not promise a result, citation or traffic.
Read the complete study, questions and source records
Primary sources referenced in preliminary source-checking
- Official OpenAI API pricing and Anthropic Claude API pricing
- NASA's Webb / K2-18 b announcement and Stevenson et al. independent study
- Google Search: AI features and your website and Google generative AI SEO guidance
- Google crawler controls and OpenAI crawler controls
- Python 3.14 free-threading documentation and NIST confidence interval guidance
- EU Commission GPAI guidelines, NeurIPS original Transformer paper and 2026 Nobel Chemistry press release
These are selected primary references, not evidence that every claim in every archived answer has been independently adjudicated. Each prompt's starting source is indexed in the appendix.
Frequently asked questions
Is ChatGPT Search more accurate than Google AI Mode?
Our 14-prompt exploratory pilot gave ChatGPT Search higher provisional scores, but ChatGPT also performed the scoring. The sample is too small and methodologically constrained to make a general accuracy ranking.
Did Google AI Mode really price a $45 workload at $270?
In one observed T11 answer, Google used a different pricing tier for GPT-6 Astra. Its calculation was consistent with that tier, but not the Standard tier explicitly requested. It is one case, not an estimated error rate.
Can I download the complete research?
Yes. The full appendix publishes the exact prompts, preliminary scoring, source records and worksheet-based response excerpts. The excerpt archive is clearly labeled as not a verified verbatim transcript of all original outputs.
Were the ratings blind or independently reviewed?
No. They were unblinded editorial ratings by ChatGPT; no independent reviewer or repeat-session study is claimed.
What should I verify most carefully in AI Search?
Anything sensitive to an exact price tier, effective date, legal responsibility, source strength, or production setting. Follow cited evidence to the specific clause or table rather than trusting a link alone.