Pay per crawl
Charging AI bots per request (HTTP 402 ideas). A publisher experiment. Most brands still want to be crawled for free.
If you charge everyone, you might fall out of answers.
If you charge scrapers and allow named lab bots, that’s a policy, not a revenue plan yet.
Examples
- A news site meters Bytespider and allows OAI-SearchBot. A reasonable stance. Metering Googlebot is how you leave Search.
Related terms
Bots from model labs and answer engines that fetch pages for training, retrieval, or user-initiated reading.
robots.txt is the fetch permission file at the site root. AI crawlers read it too. Misconfigure it and you vanish from training, search, or both.
A protocol riff on HTTP 402 so agents can pay for resources. Experimental economics for crawling and APIs.
FAQ
Should SaaS charge GPTBot? +
Usually no. You want the citation. Publishers with costly archives are the ones experimenting.
Track this in Reddex
See Reddit threads and AI answers for your brand in one place. Start with a free analysis.
Get started