Free tool
robots.txt Analyzer
Your robots.txt controls which crawlers can read your site. See exactly which AI engines (GPTBot, PerplexityBot, ClaudeBot, Google-Extended) are allowed or blocked — and whether your sitemap is declared.
Related tools
llms.txt Generator
Generate a spec-compliant llms.txt for your site in one click.
AI Visibility Audit
Score how well AI engines can find, read and cite your site.
Crawlability / Technical SEO
Check canonicals, meta robots, redirects and hreflang.
GEO Readiness Report
One-pass composite: audit + robots + schema + crawl + headers.
Need help implementing the fixes?
devcrafthouse.co.za is a full-service web development agency. We build and maintain high-performance, AI-optimised websites for businesses in South Africa and beyond.
Contact devcrafthouse.co.za