Marketplace / Safety

Crawler & Robots

Fetch any page exactly the way search and AI bots do and see what they can actually reach. Surfaces robots.txt blocks, meta-robots directives, blocked resources and status codes — the reachability signals that decide whether you get indexed. Runs on withConflux infrastructure.

urlno signup for end-usersEU-hosted
Embeds on your site, themeable to your brand:
Run a url check
Drop a file or paste a URL, then run
Check now
Powered by withConflux
<script src="https://withconflux.com/embed/v1.js"
  data-addon="crawler-robots"
  data-key="wcx_pub_live_…"
  data-theme="light"
  data-lang="auto"></script>
One snippet. Public embed key (not your secret key), locked to your domain, bot-protected.
What it returns
robots.txt & meta-robots rules
Blocked resources
Status codes & redirects
Per-bot view (Googlebot, AdsBot, AI)
€15 one-time · lifetime
  • Pay once: lifetime access, no renewal
  • Unlimited checks, on our infrastructure
  • Widget and API, one embed
  • Light usage counter (abuse protection)
Where results go
We use a single analytics cookie to understand traffic to this site. Nothing loads until you choose. See our Privacy Policy.