What decides whether AI can see you
These are the checks in your report that are about AI. The rest of the report covers the basics that matter to people and search engines too.
Headline structure
Whether the page has one clear H1 telling search engines what it is about.
Image alt text
How many images are invisible to screen readers, to search, and to AI assistants.
Business markup
Whether structured data tells search engines and assistants your address, hours and service area.
AI crawlers
Whether robots.txt shuts out GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended or CCBot.
llms.txt
Whether the site publishes the plain-text summary written for AI assistants.
The crawlers we look for
We read your robots.txt the way these crawlers do — the rules naming each one, or the catch-all rules if none do — and report any that are shut out of the whole site:
- GPTBot
- OAI-SearchBot
- ClaudeBot
- PerplexityBot
- Google-Extended
- CCBot
Blocking a single folder is a deliberate choice and is not flagged. Blocking the whole site is.
On Curb
Every Curb site lets AI crawlers in, on every plan, and publishes an llms.txt, a facts.json, and its own MCP server — so an assistant can look up your services and send you an enquiry directly, not just read about you. All of it is generated from one record of what is true about your business, so it never contradicts the pages. Get your site AI-ready.
Questions about AI visibility
Usually nobody chose to. Some hosts, security plugins and CDN settings add AI crawler blocks by default, and the owner only finds out when an assistant cannot say anything about them.
If you publish work you do not want used to train models, blocking training crawlers like CCBot or Google-Extended is a reasonable choice. Blocking the crawlers assistants use to answer questions — OAI-SearchBot, PerplexityBot — mostly just keeps you out of the answers.
A plain-text file at the root of a site, written for AI assistants: what the business is, what it offers, and where the important pages are. It saves an assistant from guessing from page markup.
No. Nobody can promise what an assistant will say. These checks tell you whether you are readable at all; what the assistant does with it depends on your reviews, your reputation and the question asked.
Keep your site. Make it readable by AI.
We move your current site onto a stack AI assistants can read. Nothing goes live until you approve it.
