robots.txt Generator
Build a robots.txt file for WordPress, Blogger or any other site in a few clicks.
Get pages found, crawled and indexed.
No sign-up. Most tools run in your browser; site checkers read a page once and store nothing.
Build a robots.txt file for WordPress, Blogger or any other site in a few clicks.
Crawl from your homepage or paste a URL list to get a clean sitemap.xml.
Paste a list of URLs, one per line.
Make an IndexNow key in your browser or use the one you have, check that the key file answers on your site, and send changed URLs to Bing, Yandex, Naver…
Load the robots.txt of any site or paste a new version, then test a URL or path for Googlebot, Bingbot, Applebot, AI search bots and AI training bots.
Type a web address to see the HTML the server sends, before JavaScript runs, with line numbers, search, line wrap and copy.
Read your sitemap to build an llms.txt file with the title and description of each page, check an existing llms.txt for format mistakes, and choose per crawler…
List the language and country versions of a page to get hreflang tags as HTML for the head, an HTTP Link header or XML sitemap entries.
Build a Search Console regex filter from a preset: question queries, long queries of at least N words, your brand with typos, a list of words or a folder of pages.
No tool matches your search.