Projects using Claude in BengaluruProjects using Claude in Bengaluru
Cover image for CrawlX
CrawlX crawls a website, surfaces
CrawlX CrawlX crawls a website, surfaces its technical SEO problems, and explains each one in plain language with a suggested fix. Cloud-native, fast, and built for teams. I built it from a blank page: product, brand, and engineering. The gap The technical SEO crawler market is led by Screaming Frog, a desktop tool with a dated interface, no AI, and a one-person, one-machine model. No sharing, no assigning issues, no client portals. At the other end, enterprise crawlers like Lumar are fast but cost $15k to $50k a year. That left a clear gap: enterprise speed, a modern interface, AI built in, and pricing a small team can actually afford. Positioning Linear meets Datadog for SEO. A crawler that does not just hand you a list of errors. It tells you what each one means and how to fix it. What I built 1. Product, engineering, and brand, end to end. 2. 400+ SEO checks across 12 analysis modules 3. An AI layer on the Claude API: plain-English explanations, auto-generated fix code, and a conversational assistant you can ask about your own crawl 4. Six core screens, including a live crawl view, a Kanban issue explorer, a site-architecture visualizer, and a white-label report builder 5. An AI search-readiness module that checks whether pages are built for AI answer engines, a gap no competitor covers 6. Rust crawl workers hitting 350 to 450 URLs per second, matching enterprise tools, with ClickHouse handling billion-row crawl data What makes it different 1. AI intelligence no legacy crawler offers: explanations, fix code, and a chat assistant, not static hints 2. Enterprise-class crawl speed at small-team pricing 3. Team-first: assign issues to developers, share client portals, comment in place, instead of one license per machine 4. Built into the pipeline: CI/CD on GitHub Actions and Vercel catches SEO regressions before they ship
0
57