Configure directives
Set your options and click Generate Tag// generate meta robots directives instantly
Build meta robots tag values for indexing, following, snippets, and image/video previews. Generate correct directives for Google, Bing, and all crawlers.
Configure directives
Set your options and click Generate TagSelect "All Bots" for a general tag, or pick a specific crawler like Googlebot or Bingbot.
Toggle indexing, following, archiving, snippet length, and image/video preview controls.
Click Generate Tag, then copy the full meta tag or just the content value into your HTML head.
The meta robots tag is an HTML element that tells search engine crawlers how to handle a page — whether to index it, follow links, show a snippet, or cache it. This builder generates the correct syntax for all standard directives, including Google-specific extensions.
A meta robots tag is placed in the <head> of an HTML page to instruct search engine crawlers on how to index and treat the page. Common directives include noindex (don't index), nofollow (don't follow links), and nosnippet (don't show a text snippet in results).
robots.txt controls crawler access — it tells bots which URLs they can fetch. Meta robots tags control indexing behavior for pages that are already crawled. A page blocked in robots.txt won't be crawled at all; a page with noindex can be crawled but won't appear in search results.
The max-snippet directive (Google-specific) lets you set the maximum character length of a text snippet shown in search results. Use -1 for no limit, 0 to disallow snippets entirely, or a positive number to cap the length.
Yes. Instead of using name="robots", you can use name="googlebot" or name="bingbot" to set directives for specific crawlers only. This lets you allow Bing to index a page while blocking Google, for example.
The noarchive directive tells search engines not to show a cached copy of the page. Users clicking "Cached" in Google results will see a message that no cached version is available. Useful for frequently updated pages or sensitive content.
This Google directive controls the maximum size of image previews shown in search results. Options are none (no image preview), standard (default size), or large (larger preview, used for Google Discover and featured images).
No — noindex and nofollow are independent directives. A noindex page can still pass link equity if you use follow. However, because Google won't index the page, the practical impact on link equity through that page is limited.
The meta robots tag must be placed inside the <head> section of your HTML document, before the closing </head> tag. Bots read it during the initial crawl — placing it elsewhere (like the body) means it may be ignored.
The meta robots tag is a snippet of HTML code placed in the <head> section of a webpage that communicates instructions directly to search engine crawlers. Unlike the robots.txt file — which gates crawler access at the server level — the meta robots tag works at the page level, giving you granular control over how each individual URL is treated in search engine indices.
Its basic form looks like this: <meta name="robots" content="index, follow">. The name attribute specifies which bot(s) the directive applies to, and the content attribute lists one or more directives separated by commas. When no meta robots tag is present, search engines default to treating the page as index, follow.
💡 Looking for SEO-optimized themes to give your site a competitive edge? MonsterONE offers unlimited downloads of SEO-ready website templates, landing pages, and UI kits — worth checking out if you're building or redesigning.
Understanding each directive is key to using this tool effectively. Here's a breakdown of the most important ones:
noindex is one of the most powerful directives, used to hide thin content, duplicate pages, login screens, and staging environments from search results.nofollow prevents link equity from passing through links on the page, often used on user-generated content pages or comment sections.Google introduced a set of advanced content preview directives that give publishers fine-grained control over how their content is displayed in search results — particularly relevant for publishers concerned about Google's rich snippets consuming too much of their content without driving clicks.
-1 to allow unlimited snippets, 0 to prohibit snippets entirely, or a positive integer to cap the length.none, standard, and large. Setting large enables eligibility for Google Discover and larger image panels.-1 for no limit or a positive integer to cap preview duration.While name="robots" applies to all compliant crawlers, you can target specific bots by name. For example, name="googlebot" sends directives exclusively to Google's crawler, while name="bingbot" targets Microsoft's. This is useful when you want different search engines to treat a page differently — allowing Bing to cache a page while preventing Google from showing a cached version, for instance.
You can also stack multiple meta robots tags on a single page — one for all bots and additional ones for specific crawlers. Crawlers that honor these tags will use the most specific directive available for them. Custom crawler names can be used for specialized bots that support the robots meta tag specification.
This is one of the most common points of confusion in SEO. The key rule: use robots.txt to prevent crawling, use meta robots to prevent indexing. If a URL is disallowed in robots.txt, Google won't crawl it — but it may still index it based on external links. If you truly want a page out of the index, you must be able to serve the page (so Google can read the noindex tag) and set noindex in the meta robots tag.
Common mistakes include blocking a page in both robots.txt and with noindex — the noindex tag can't be read if the crawler is blocked from accessing the page. Always test your configuration using Google Search Console's URL Inspection tool after deploying changes.
For non-HTML files like PDFs, images, and videos — where you can't add a <meta> tag — you can deliver the same directives via the X-Robots-Tag HTTP response header. The syntax mirrors the meta tag content value: X-Robots-Tag: noindex, nofollow. This builder generates the equivalent HTTP header alongside the meta tag for convenience.
The noindex directive is appropriate in several common situations: paginated search results pages, thin or low-quality content pages, internal search results, staging and development environments, user account and settings pages, thank-you or confirmation pages, and any page you actively want to exclude from search results. For e-commerce sites, filtered category pages with little unique content are prime candidates for noindex.
Be cautious: over-applying noindex can significantly reduce a site's overall indexed footprint and organic visibility. Always audit before deploying noindex at scale, and use Google Search Console to monitor de-indexation.