Indexability Checker Tool Build

I built the Indexability Checker as a focused technical SEO utility for answering one practical question: does this URL look eligible to be crawled and indexed based on visible page-level signals?

The tool is intentionally careful. It does not claim to know whether Google has indexed a page. Instead, it checks the signals a public fetcher can responsibly inspect and turns them into a plain-English verdict.

Final HTTP status and redirect path

robots.txt access for the selected crawler

Meta robots noindex directives

X-Robots-Tag noindex headers

Canonical tags and canonicalized-elsewhere signals

Crawl and index eligibility warnings in priority order

The public page runs on Astro inside bree-sharp.com. The server-side analysis runs through a dedicated Next.js API deployed on Vercel, because browser-only checks cannot reliably fetch cross-origin HTML, headers, redirect chains, and robots.txt files.

The API follows redirects, records the final response, fetches and parses robots.txt, inspects HTML for meta robots and canonical tags, reads X-Robots-Tag headers, and returns a concise verdict with supporting evidence.

Production also blocks localhost, private IPs, and internal-network targets so the endpoint cannot be misused as a private-address fetch proxy. Local development stays flexible, but the deployed tool is intentionally stricter.

The result is a fast URL-level diagnostic that sits between the robots.txt checker and sitemap validator. Robots.txt answers whether a crawler may request a URL. The sitemap validator checks what a site asks crawlers to discover. The Indexability Checker inspects the page-level signals that decide whether that discovered URL is a real indexing candidate.