Why isn't my page showing up in Google even though I didn't add a noindex tag?

A noindex directive isn't the only way a page gets excluded - it could also be disallowed in robots.txt (which stops Google from crawling it at all), too new to be crawled yet, or excluded for quality reasons unrelated to any tag. This tool checks the tag-based signals directly so you can rule those out first.

What's the difference between a robots.txt disallow and a noindex tag?

A robots.txt disallow blocks crawling - search engines won't fetch the page's content at all, but the URL itself can still appear in results (without a title or snippet) if other pages link to it. A noindex tag requires the opposite: the page must be crawlable so the search engine can see the tag, and it then guarantees the page is left out of results entirely. If you actually want a page gone from search results, noindex (and letting it stay crawlable) is the reliable path, not robots.txt.

Why do I need to paste response headers separately from the HTML?

An X-Robots-Tag noindex directive lives in the HTTP response headers, not in the page's HTML, so it's invisible if you only look at "View Page Source." It's commonly used to noindex non-HTML files like PDFs where there's no to put a meta tag in. Copy the headers from your browser's Network tab or run `curl -I` on the URL to check this signal.

Does a meta googlebot tag override a meta robots tag?

Yes, for Google specifically. When both are present, Google follows the more specific directive for its own crawler, while other search engines that don't recognize that tag still follow the general one. This tool checks both and reports them separately.

Noindex Checker - Find Every Way a Page Is Blocked From Search

Overview

Introduction

A page can be silently excluded from search results in more than one way, and none of them are visible just by looking at the rendered page in a browser: a noindex meta tag buried in the <head>, an X-Robots-Tag header that only shows up in the raw HTTP response, or a robots.txt rule that blocks the crawler before it ever sees the page's content.

This tool checks all three signals in one place and gives a single indexable/blocked verdict, explaining exactly which signal is responsible when a page is blocked, rather than leaving you to hunt through the HTML, headers, and robots.txt separately.

What Is Noindex Checker?

A checker that inspects a page's <meta name="robots"> and <meta name="googlebot"> tags for a noindex directive, an optional pasted X-Robots-Tag response header for the same, and an optional pasted robots.txt for a disallow rule matching the page's path.

It works two ways for the HTML and headers: fetching a URL directly from your browser when the target allows cross-origin requests, or reading content you paste in yourself, which works regardless of CORS.

How Noindex Checker Works

The HTML is parsed with the browser's own DOMParser to read the robots and googlebot meta tags; pasted response headers are parsed for an X-Robots-Tag line; pasted robots.txt content is parsed into User-agent groups and tested against the page's path using the same longest-match precedence rules search engines document.

Each signal is reported individually with a good/warning/missing status and an explanation, then rolled up into one indexable/blocked verdict that names exactly which signal(s) caused a block.

When To Use Noindex Checker

Right after a page unexpectedly drops out of search results, to quickly rule in or out a noindex tag, header, or robots.txt change as the cause.

Before launching a page you expect to be indexed, to confirm a staging-environment noindex tag or header didn't accidentally make it into production.

While auditing a site migration, since CMS platform changes commonly introduce an unintended sitewide noindex or X-Robots-Tag header.

Often used alongside Robots.txt Tester, Canonical URL Checker and Meta Tag Checker.

Features

Advantages

Checks all three major exclusion signals (meta robots, meta googlebot, X-Robots-Tag) in one pass instead of requiring three separate lookups.
Explains the real-world difference between a robots.txt disallow and a noindex directive instead of treating them as interchangeable.
Works on any page via the paste-source fallback, not just pages that happen to allow cross-origin fetches.

Limitations

A live fetch only works when the target site sends permissive CORS headers, which most ordinary websites don't; use the paste fallback for the HTML, headers, and robots.txt inputs instead.
Response headers generally can't be read from a cross-origin fetch() at all due to browser security restrictions, so the X-Robots-Tag check almost always needs the pasted-headers path, not the fetch path.
This tool reports what the signals say, not whether Google has actually crawled or indexed the page - a newly noindex-tagged page can remain indexed for a while until it's recrawled.

Examples

A page correctly blocked from indexing

Input

<meta name="robots" content="noindex, follow">

Output

Blocked from indexing
Blocked by: <meta name="robots"> noindex

The noindex token excludes the page from search results, while "follow" still lets its outbound links pass ranking signal.

A page with no HTML-level block but disallowed in robots.txt

Input

HTML: no robots meta tag
robots.txt: Disallow: /private/
Path tested: /private/report

Output

Indexable (blocking signals: none)
robots.txt (/private/report): warning - "Disallow: /private/" matched

No noindex directive was found, so the verdict is nuanced: the page is disallowed from crawling, which is flagged as a warning, but that alone doesn't guarantee removal from search results the way noindex would.

Best Practices & Notes

Best Practices

If you actually want a page removed from search results, use a noindex directive and make sure the page stays crawlable (not disallowed in robots.txt), since Google can't see a noindex tag on a page it's blocked from fetching.
Check X-Robots-Tag headers on non-HTML files (PDFs, images, XML) specifically, since a meta tag isn't an option there and it's an easy thing to forget when auditing a site.
Re-run this check after any CMS, CDN, or reverse-proxy change, since those layers commonly inject or strip X-Robots-Tag headers without touching the page's HTML at all.

Developer Notes

The verdict intentionally treats a robots.txt disallow as a blocking signal alongside noindex, even though the two mechanisms behave differently (disallow blocks crawling, not indexing outright) - the note text explains the distinction so the verdict stays honest rather than collapsing a real nuance into a flat true/false.

Noindex Checker Use Cases

Diagnosing why a specific page has dropped out of, or never appeared in, search results
Confirming a staging or password-protected environment's noindex tag didn't leak into a production deploy
Auditing X-Robots-Tag headers on downloadable files (PDFs, spreadsheets) that can't carry a meta tag

Common Mistakes

Assuming a robots.txt disallow is equivalent to noindex - a disallowed URL can still show up in search results without a snippet if it's linked from elsewhere.
Only checking the rendered HTML and missing an X-Robots-Tag header, since that signal is invisible in "View Page Source" and requires inspecting the actual HTTP response.
Forgetting that a <meta name="googlebot"> tag can override a more permissive <meta name="robots"> tag specifically for Google's crawler.

Tips

Copy response headers from your browser DevTools' Network tab (click the request, then "Headers") or run `curl -I <url>` to get the X-Robots-Tag input this tool needs, since a live fetch usually can't read it due to CORS.

Noindex Checker