Official website
Jacob Whitmore
Search is changing. I measure what happens next.
I research search, AI and how people find things online. Every study is dated, every dataset is public, and every prediction gets marked right or wrong.

- 12 yearsin SEO, across agency, freelance and in-house roles
- Open dataa CSV with every study, free to reuse with a link
- On the recorddated predictions, scored right or wrong, misses left up
- MonthlyUK AI-crawler snapshots, running since October 2026

01 About
Hello, I’m Jacob.
I’ve worked in SEO for around 12 years. Now I study what happens when search changes.
Who gets discovered, which businesses AI tools recommend, and what the evidence says about where search is heading. I test it with small, repeatable studies and publish the data behind each one.
This site is where it all goes: the findings, the method, and the calls I get wrong.
02 Background
Twelve years in search.
Three kinds of SEO work, then a move into public research. The constant: watching how search decides who gets seen.
- Where it started
Agency
Client SEO across many sites at once. The best place to learn how search behaves.
- Then
Freelance
Working directly with businesses on their own search visibility.
- Recent years
In-house
Search in one of the most competitive corners of UK search, watched up close.
- Since October 2026
Independent research
Dated studies, open datasets and a public prediction ledger, all on this site.
03 Research and insights
Original research. Data attached.
Each study states when the data was collected, the sample, the method and its limits. The charts are drawn straight from the published spreadsheets.
Which AI crawlers do the UK’s biggest news sites block?
9 of 10 block at least one AI crawler. They block the bots that train AI models far more often than the bots that fetch pages for AI answers.
| Site | GPTBot | ClaudeBot | anthropic-ai | Google-Extended | CCBot | Applebot-Extended | Bytespider | Meta-ExternalAgent | Amazonbot | OAI-SearchBot | ChatGPT-User | Claude-SearchBot | Claude-User | PerplexityBot | Perplexity-User | Blocked |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Daily Mail | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | 15/15 |
| The Telegraph | Blocked | Blocked | Blocked | Not blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | 14/15 |
| BBC | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Not blocked | Not blocked | Blocked | Blocked | 13/15 |
| The Sun | Not blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Not blocked | Not blocked | Blocked | Blocked | Blocked | Blocked | 12/15 |
| Sky News | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Not blocked | Not blocked | Not blocked | Not blocked | Blocked | Not blocked | 10/15 |
| The Guardian | Not blocked | Blocked | Blocked | Not blocked | Blocked | Blocked | Blocked | Blocked | Blocked | Not blocked | Not blocked | Blocked | Blocked | Blocked | Not blocked | 10/15 |
| Yahoo! (UK news) | Blocked | Blocked | Blocked | Blocked | Blocked | Not blocked | Blocked | Not blocked | Not blocked | Blocked | Blocked | Blocked | Not blocked | Blocked | Not blocked | 10/15 |
| Mirror | Blocked | Blocked | Blocked | Not blocked | Blocked | Blocked | Not blocked | Blocked | Not blocked | Blocked | Not blocked | Not blocked | Not blocked | Blocked | Not blocked | 8/15 |
| Daily Express | Blocked | Blocked | Blocked | Not blocked | Blocked | Blocked | Not blocked | Not blocked | Not blocked | Blocked | Not blocked | Not blocked | Not blocked | Blocked | Not blocked | 7/15 |
| The Independent | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | Not blocked | 0/15 |
- Sample
- 10 biggest UK newsbrands × 15 AI crawlers. 150 checks a month.
- Method
- Each site’s robots.txt is read and saved on the 2nd of every month. What counts as blocked is defined in the method note.
Do the AI search statistics everyone repeats hold up?
None of the 32 passes all five checks, and none is about the UK.
- Sample
- 32 statistics that appear on at least 3 of 20 roundup pages.
- Method
- Each statistic traced to the document it came from, then checked for five things: when, how many, where, how, and whether the data is published.
Do AI answers say when a “current” figure is from?
7 of 40 answers didn’t say when the figure was from. All 7 came from Google’s two engines.
- Sample
- 40 answers: 10 UK figure questions on 4 AI engines.
- Method
- Every question asked for a figure “now” or “today”, and every figure has an official source that publishes it with a date.
04 Predictions and observations
Called early. Scored in public.
Each prediction is written down with a date and a test before the result is in. On the check date it gets marked. Wrong calls stay up, in red, unedited.
- Right
- 2
- Wrong
- 1
- Still open
- 29
- Couldn’t score
- 0
P19 Right
After the follow-up 'When is that figure from? Give the month or period and the official source.', at least 95% of AI answers will state a period for the figure.
Scored 18 Nov 2026: hit. 80 of 80 arm D second answers stated a period (75 of 80 if a year is required; the registered test does not require one). Results: https://jacobwhitmore.com/date-follow-up-results/
Due next
- P17On the first Russell Group robots.txt snapshot, a smaller share of the 24 universities will block GPTBot than the UK's ten biggest newsbrands did (7 of 10).
- P18On the first Russell Group snapshot, universities will block training crawlers more often than retrieval crawlers, by at least 10 percentage points.
- P1On a rerun of the same ten UK newsbrands and 15 crawlers, at least 7 of 10 will still block training crawlers more often than user-triggered fetchers.
05 Current interests
What I’m looking at now.
Six threads that most of the work hangs off. Each one starts with the piece that explains it best.
06 Selected writing
Worth reading first.
Picked for what they settle, not how new they are.
Does publishing original research get you cited by AI?
No controlled study shows original research raises AI citations. Why it should help AI use your work, why credit is less certain, and the test this site will score on 2 Jan 2027.
Will AI agents do your shopping? Three of the UK’s biggest retail sites have said no in writing
Checked 12 November 2026. Amazon, eBay and Vinted tell AI shopping agents to stay out of checkout. Seven of the UK's ten biggest retail sites say nothing. What the files and one court ruling show.
How do ChatGPT, Claude, Gemini, Perplexity and Copilot choose their sources? What the companies actually say
Checked 8 Nov 2026: what OpenAI, Anthropic, Google, Perplexity and Microsoft say about how their AI answers find and pick sources. Only one explains how it chooses.
How many times should you run an AI prompt before you trust the number?
To pin an AI brand-mention rate to within ±10 points you need about 97 runs of one prompt. To show a change worked, about 388 per version. The maths and a free calculator.
Do backlinks get you cited in AI answers? What 16 studies actually measured
Checked 30 October 2026. 16 studies on links, authority and rankings vs AI citations. All correlational, none with raw data, one naming a country. Brand mentions beat links.
Gartner said search volume would drop 25% by 2026. Can it be scored?
Gartner's 2024 call that search volume would fall 25% by 2026, checked in October 2026. It can't be scored as written. On the closest public data, it looks like a miss.
07 Press and citations
Findings you can quote.
Each figure is one dated reading with the data behind it. Please quote the date with the number.
Nine of the UK’s ten biggest newsbrands block at least one AI crawler in robots.txt.
None of the UK’s ten biggest supermarkets blocks any of the same 15 AI crawlers.
None of 32 widely repeated AI search statistics passes all five source checks.
08 Contact
Get in touch.
contact@jacobwhitmore.comFor press, research questions or corrections, email is fastest. New results go up on LinkedIn first.