WPMozo Add-ons and tools for WordPress, Block Editor, Elementor and WooCommerce Explore add-ons
⚡ SEO & Schema · Client-side · No data sent to server

robots.txt Tester

Paste your robots.txt, choose a crawler, and test any URL or path to see whether it is allowed or blocked, and exactly which rule decided it. Uses the same longest-match and allow-wins logic Google applies. Runs entirely in your browser.

robots.txt

URLs or paths to test

Result
How the decision is made. Among the rules for the matching user-agent, the longest matching path wins; if an Allow and a Disallow tie, Allow wins. Wildcards * and end-anchor $ are supported. robots.txt controls crawling, not indexing: a blocked URL can still appear in results if linked elsewhere. Build a file from scratch with the robots.txt Builder. Everything is computed in your browser.

Related tools

Indexability Checker Paste a page’s HTML to see if search engines can index it: meta robots, X-Robots-Tag, canonical, and meta refresh, with a clear verdict. robots.txt Builder Build a robots.txt file with common rules. Canonical Tag Checker Check the canonical tag in pasted HTML: present, self-referencing, absolute, and consistent, with common mistakes flagged. Link Extractor Extract every link from pasted HTML and split them into internal, external, and nofollow, with anchor text and counts. Google SERP Snippet Preview Preview how your title and meta description appear in Google results on desktop and mobile, with length warnings. Social Media Preview Preview Facebook, X, and LinkedIn share cards from your Open Graph and Twitter tags, and spot any missing ones.

Guides

DiviExtended Premium Divi child themes and plugins View Divi Plugins