Check robots.txt, sitemap and llms.txt
The most important free LLM readiness check: robots.txt, sitemap and llms.txt.
This module checks whether the domain provides a robots.txt, a sitemap and LLM-relevant files such as llms.txt or llms-full.txt. It also detects whether known AI bots are blocked, allowed or not mentioned in robots.txt.
This section is exactly what makes an AI visibility check different from an ordinary website scan. If AI visibility matters, you need to know whether content is accessible to ChatGPT Search, Claude Search, Perplexity or other crawlers at all.
- status of robots.txt, sitemap and llms.txt
- initial AI crawler access assessment
- warning for missing or conflicting signals
- bot-level rule analysis by crawler and purpose
- recommendations for separate search / user / training strategies
- matching with logs and real visibility experience
If you already checked a URL on the main page, CheckCom shows a live or demo evaluation for this exact module here. Without a previous check, a sample result is loaded automatically.
Need more data, real model tests and full reports?
The full version adds real multi-LLM tests, deeper SERP and competitor analysis, export formats, action plans and individual interpretation.