Don’t guess. Measure.
Jacob Whitmore
Independent research into the future of search.
Search is changing. I measure what happens next. Original research, public datasets and predictions scored in public.
- 12 yearsin SEO, across agency, freelance and in-house roles
- Open dataa CSV with every study, free to reuse with a link
- On the recorddated predictions, scored right or wrong, misses left up
- MonthlyUK AI-crawler snapshots, running since October 2026
01 Latest research
Original research. Data attached.
Each study states when the data was collected, the sample, the method and its limits. The data file is published with it.
Google answered 40 of 40 UK questions before the first link
- Question
- What does Google put above the first ordinary link on everyday UK questions?
- Collected
- 18 November 2026, google.co.uk, desktop
- Sample
- 40 questions in four groups of ten: quick facts, how-to, health and money
- Finding
- An AI Overview on 38, Google’s own answer box on 3. On 22 of 40, no ordinary link was visible without scrolling.
- Limits
- A pilot. One location, one browser, one morning. It shows what Google displayed, not what searchers clicked.
- Next
- Same 40 questions, around 18 December 2026
Which AI crawlers do the UK’s biggest news sites block?
9 of 10 block at least one AI crawler. They block the bots that train AI models far more often than the bots that fetch pages for AI answers.
| Site | GPTBot | ClaudeBot | anthropic-ai | Google-Extended | CCBot | Applebot-Extended | Bytespider | Meta-ExternalAgent | Amazonbot | OAI-SearchBot | ChatGPT-User | Claude-SearchBot | Claude-User | PerplexityBot | Perplexity-User | Blocked |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Daily Mail | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | 15/15 |
| The Telegraph | Blocked | Blocked | Blocked | Not blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | 14/15 |
| BBC | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Not blocked | Not blocked | Blocked | Blocked | 13/15 |
| The Sun | Not blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Not blocked | Not blocked | Blocked | Blocked | Blocked | Blocked | 12/15 |
| Sky News | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Not blocked | Not blocked | Not blocked | Not blocked | Blocked | Not blocked | 10/15 |
| The Guardian | Not blocked | Blocked | Blocked | Not blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Not blocked | Not blocked | Blocked | Blocked | Blocked | Not blocked | 10/15 |
| Yahoo! (UK news) | Blocked | Blocked | Blocked | Blocked | Blocked | Not blocked | Blocked | Not blocked | Not blocked | Blocked | Blocked | Blocked | Not blocked | Blocked | Not blocked | 10/15 |
| Mirror | Blocked | Blocked | Blocked | Not blocked | Blocked | Blocked | Not blocked | Blocked | Not blocked | Blocked | Not blocked | Not blocked | Not blocked | Blocked | Not blocked | 8/15 |
| Daily Express | Blocked | Blocked | Blocked | Not blocked | Blocked | Blocked | Not blocked | Not blocked | Not blocked | Blocked | Not blocked | Not blocked | Not blocked | Blocked | Not blocked | 7/15 |
| The Independent | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | 0/15 |
- Sample
- 10 biggest UK newsbrands × 15 AI crawlers. 150 checks a month.
- Method
- Each site’s robots.txt is read and saved on the 2nd of every month. What counts as blocked is defined in the method note.
Do the AI search statistics everyone repeats hold up?
None of the 32 passes all five checks, and none is about the UK.
- Sample
- 32 statistics that appear on at least 3 of 20 roundup pages.
- Method
- Each statistic traced to the document it came from, then checked for five things: when, how many, where, how, and whether the data is published.
Do AI answers say when a “current” figure is from?
7 of 40 answers didn’t say when the figure was from. All 7 came from Google’s two engines.
- Sample
- 40 answers: 10 UK figure questions on 4 AI engines.
- Method
- Every question asked for a figure “now” or “today”, and every figure has an official source that publishes it with a date.
02 Findings
What the evidence says so far.
Each figure is one dated reading with the data behind it. Free to quote with the date and a link.
Nine of the UK’s ten biggest newsbrands block at least one AI crawler in robots.txt.
None of the UK’s ten biggest supermarkets blocks any of the same 15 AI crawlers.
None of 32 widely repeated AI search statistics passes all five source checks.
03 The Jacob Whitmore Index
The numbers, in one table.
A fixed set of readings on how UK websites and AI search treat each other. Each has the date it was measured and the date it gets measured again.
| Measure | Latest reading | Measured | Next reading |
|---|---|---|---|
| JWI-1 · AI crawler checks blocked by the UK’s ten biggest newsbrands | 99 of 150 | 2 Nov 2026 | 2 Dec 2026 |
| JWI-3 · The same checks blocked by the UK’s ten biggest supermarkets | 0 of 150 | 10 Nov 2026 | 10 Feb 2027 |
| JWI-4 · The same checks blocked by the UK’s ten most-visited retail sites | 27 of 150 | 12 Nov 2026 | 12 Feb 2027 |
| JWI-8 · Most-repeated AI search statistics that pass all five source checks | 0 of 32 | 29 Oct 2026 | 29 Jan 2027 |
| JWI-9 · AI visibility trackers that publish evidence their numbers are accurate | 0 of 13 | 30 Oct 2026 | 30 Apr 2027 |
| JWI-11 · Everyday UK questions Google answered above the first ordinary link | 40 of 40 | 18 Nov 2026 | 18 Dec 2026 |
Six of the twelve current readings, as published on 19 November 2026. A reading is never edited. A new study adds a new one.
04 Public Prediction Ledger
Called early. Scored in public.
Each prediction is written down with a date and a test before the result is in. On the check date it gets marked. Wrong calls stay up, in red, unedited.
- Right
- 2
- Wrong
- 1
- Still open
- 30
- Couldn’t score
- 0
P19 Right
After the follow-up 'When is that figure from? Give the month or period and the official source.', at least 95% of AI answers will state a period for the figure.
Scored 18 Nov 2026: hit. 80 of 80 arm D second answers stated a period (75 of 80 if a year is required; the registered test does not require one). Results: https://jacobwhitmore.com/date-follow-up-results/
Due next
- P17On the first Russell Group robots.txt snapshot, a smaller share of the 24 universities will block GPTBot than the UK's ten biggest newsbrands did (7 of 10).
- P18On the first Russell Group snapshot, universities will block training crawlers more often than retrieval crawlers, by at least 10 percentage points.
- P38On the 18 December 2026 rerun of the same 40 UK questions, Google will show an answer (an AI Overview or its own answer box) above the first ordinary link on at least 38 of the 40.
05 Current investigations
What I’m measuring next.
Work that is planned or under way, with the date each result is due. Nothing here has a result yet.
| Investigation | Where it stands | Due |
|---|---|---|
| UK news sites and AI crawlers, reading three | Monthly series. Two readings published. | 2 Dec 2026 |
| Russell Group universities and AI crawlers | First snapshot. Predictions P17 and P18 get checked against it. | 2 Dec 2026 |
| Undocumented AI crawler rules | Plan published. First reading runs with the December news snapshot. | 2 Dec 2026 |
| Google answers above the first link, reading two | Pilot published 18 Nov 2026. Same 40 questions again. | Around 18 Dec 2026 |
| The industry’s 2026 SEO predictions, scored | Plan published. Pages collected 1 Dec 2026, scored in January. | By 31 Jan 2027 |
06 Selected analysis
Worth reading first.
Picked for what they settle, not how new they are.
Does publishing original research get you cited by AI?
No controlled study shows original research raises AI citations. Why it should help AI use your work, why credit is less certain, and the test this site will score on 2 Jan 2027.
Will AI agents do your shopping? Three of the UK’s biggest retail sites have said no in writing
Checked 12 November 2026. Amazon, eBay and Vinted tell AI shopping agents to stay out of checkout. Seven of the UK's ten biggest retail sites say nothing. What the files and one court ruling show.
How do ChatGPT, Claude, Gemini, Perplexity and Copilot choose their sources? What the companies actually say
Checked 8 Nov 2026: what OpenAI, Anthropic, Google, Perplexity and Microsoft say about how their AI answers find and pick sources. Only one explains how it chooses.
How many times should you run an AI prompt before you trust the number?
To pin an AI brand-mention rate to within ±10 points you need about 97 runs of one prompt. To show a change worked, about 388 per version. The maths and a free calculator.
Do backlinks get you cited in AI answers? What 16 studies actually measured
Checked 30 October 2026. 16 studies on links, authority and rankings vs AI citations. All correlational, none with raw data, one naming a country. Brand mentions beat links.
Gartner said search volume would drop 25% by 2026. Can it be scored?
Gartner's 2024 call that search volume would fall 25% by 2026, checked in October 2026. It can't be scored as written. On the closest public data, it looks like a miss.
07 Press and media
For journalists.
Bios, a press photo, quotable findings and citation guidance are in the media centre. Mistakes get corrected in public under the editorial policy.
08 About Jacob
Hello, I’m Jacob.
I’ve worked in SEO for around 12 years. Now I study what happens when search changes.
Who gets discovered, which businesses AI tools recommend, and what the evidence says about where search is heading. I test it with small, repeatable studies and publish the data behind each one.
This site is where it all goes: the findings, the method, and the calls I get wrong.
09 Follow the research
Follow the research.
contact@jacobwhitmore.comNew results go up on LinkedIn first. The RSS feed carries every study and analysis piece. For press, research questions or corrections, email is fastest.