Content Scrapper, AI-Assisted Tool That Eliminated Manual CMS Content Migration
Built with Claude Code to replace weeks of manual copy-pasting with an automated crawl-extract-review pipeline for hotel website migrations.
Description:
The problem: our company was migrating 25+ hotel property websites into a new CMS. Without tooling, that meant someone manually opening every single page and copy-pasting headings, body copy, promotional sections, testimonials, contact info, and every image's alt text into the new
system slow, repetitive, and error-prone at the scale of hundreds of pages per site.
I built Content Scrapper, solo, to remove that manual work entirely no team, no handoffs. I used Claude (Claude Code) as my AI pair-programmer throughout the build, from designing the section-extraction schema and adapter architecture to implementing the rendering pipeline, CLI,
and web UI which let me go from idea to a working, production-usable tool much faster, while I stayed in control of every architectural and domain decision.
What it does:
Renders every page with a real headless browser (Playwright/Chromium), since key sections (amenities, nearby attractions, blog teasers) only load client-side via AJAX a plain fetch would have missed most of the content.
Automatically detects which site template a page uses and extracts it into a consistent JSON shape (hero, content blocks, touts, testimonials, contact info, images with alt text, meta tags, JSON-LD) with a generic fallback extractor for any site that isn't a known template, so no per-site manual configuration is required.
Ships as both a CLI (crawl → extract → review) for bulk migration runs, and a lightweight internal web UI so non-technical teammates can preview scraped content or run a whole-site scrape themselves without needing to touch the terminal or ask an engineer to do it for them.
Includes a built-in review step that automatically flags missing alt text, missing meta descriptions, and pages that scraped suspiciously thin catching problems before content ever reaches the CMS.
Adds a domain allowlist and SSRF guard on the web UI, since the tool renders whatever URL is submitted in a real browser and needed to be safe to hand to the wider team.
Impact: what used to be manual, page-by-page data entry across 5+ full hotel websites became a repeatable, self-service pipeline freeing the team from copy-paste work and human error, while producing structured, review-ready JSON the CMS-integration phase can consume directly.
How it's built:
Language: TypeScript, Node.js 18+, run via tsx (no build step needed)
Browser rendering: Playwright (headless Chromium) — for JS-rendered/AJAX content
HTML parsing/extraction: Cheerio (server-side jQuery-style DOM parsing)
CLI: Commander.js
Backend/API: Express.js
Validation: Zod (site config validation)
Concurrency control: p-limit (rate-limited parallel page rendering)
Frontend: Vanilla HTML/JS single page (no framework, no build step)
Storage: Filesystem-based normalized JSON (site → pages → sections)
Architecture pattern: Adapter pattern with auto-detection (canHandle() per adapter)
Security: hostname allowlist + private-IP guard (SSRF protection)
Built with: Claude Code (Anthropic) as AI pair-programmer
Links
GitHub: https://github.com/jaiinil/content-scrapper
1
20
Project Title: Global Corporate Platform – Diageo
Role: Frontend Engineer | Team: IDX (Investis Digital)
Live Link: https://www.diageo.com/
(https://www.diageo.com/)🚀 Project Overview
Collaborated as a core member of the IDX (Investis Digital) engineering team to transform high-fidelity Figma wireframes and designs into a fully functional, global corporate platform for Diageo. My role focused on the "design-to-digital" pipeline, ensuring that the visual vision was translated into a high-performance, Sitecore-integrated reality.
🛠 Technical Stack & Tools
Design-to-Code: Figma (Asset Extraction & UI Specs)
Systems Architecture: Storybook (Component-Driven Development)
CMS Integration: Sitecore Experience Platform
Frontend Core: HTML5, CSS3 (SCSS), Modern JavaScript (ES6+)
Libraries: Bootstrap, GSAP (GreenSock) for interactive animations
Compliance: WCAG 2.1 Level AA Accessibility
🛠 My Key Contributions within the Team
Figma-to-Digital Implementation: Led the technical translation of complex Figma designs and wireframes into responsive, pixel-perfect frontend code, ensuring brand consistency across all breakpoints.
Component Development (Storybook): Built and documented modular UI components in Storybook, creating a "living" design system that served as the bridge between the design team and Sitecore developers.
Sitecore Integration: Collaborated with the engineering team to build "Sitecore-ready" modules, ensuring that the high-fidelity designs remained intact when managed by content editors.
Interactive Features: Implemented specific high-end interactions using GSAP, including scroll-triggered animations for the "Spirit of Progress" ESG sections.
Accessibility Standards: Ensured all UI components met WCAG 2.1 AA standards, focusing on inclusive design and screen reader compatibility.
📈 Team Achievement
Successfully launched a multi-region corporate platform that perfectly mirrors the original Figma design intent.
Contributed to a modular, scalable codebase that enabled the agency to manage global content updates efficiently for the client.
1
48
Project Title: Global Infrastructure Platform – AtkinsRéalis
Role: Frontend Engineer | Team: IDX (Investis Digital)
Live Link: https://www.atkinsrealis.com/
🚀 Project Overview
Collaborated with the IDX engineering team to develop the frontend for the global digital rebrand of AtkinsRéalis. The project involved a massive site migration and consolidation, requiring a highly structured, scalable frontend architecture to support their global operations across multiple sectors.
🛠 Technical Stack & Tools
Systems Architecture: Storybook (Component Library Management)
Design-to-Code: Figma (High-fidelity wireframes to UI)
Frontend Core: HTML5, CSS3 (SCSS), Modern JavaScript (ES6+)
Frameworks: Bootstrap (Custom Enterprise Grid)
Compliance: WCAG 2.1 Level AA (Global Accessibility Standards)
🛠 My Key Contributions within the Team
Figma-to-Digital Transformation: Translated complex engineering-focused Figma designs into a clean, professional, and responsive UI that reflects the firm's global authority.
Scalable Component Development: Built modular UI components in Storybook, ensuring they were flexible enough to be reused across different regions and business sectors (Nuclear, Infrastructure, etc.).
Performance Optimization: Focused on optimizing heavy media assets and complex data tables to ensure fast load times for a global audience.
Accessibility (a11y): Implemented strict WCAG 2.1 AA standards, ensuring the site is fully accessible to investors, partners, and government stakeholders.
📈 Team Achievement
Successfully supported a major global rebranding effort, launching a unified digital presence that handles high traffic and complex information architecture.
Delivered a pixel-perfect UI that maintains brand integrity across all global sub-domains.
0
28
Project Title: AgTech Innovation Platform – Invaio
Role: Frontend Engineer | Team: IDX (Investis Digital)
Live Link: https://www.invaio.com/
🚀 Project Overview
Collaborated as part of the IDX engineering team to build the digital platform for Invaio, a global leader in precision biotechnologies for agriculture. The project focused on creating a modern, high-impact site that communicates complex scientific innovation through a clean, user-centric interface.
🛠 Technical Stack & Tools
Design-to-Code: Figma (Pixel-perfect UI implementation)
Systems Architecture: Storybook (Modular Component Library)
Frontend Core: HTML5, CSS3 (SCSS), Modern JavaScript (ES6+)
Media Handling: Advanced Video Integration (Background & Popups)
Compliance: WCAG 2.1 Level AA Accessibility
🛠 My Key Contributions within the Team
Figma-to-Digital Execution: Transformed high-fidelity Figma designs into a responsive, fluid frontend, ensuring that the scientific and tech-forward brand identity was maintained across all devices.
Modular Component Development: Built and documented a library of reusable UI components in Storybook, allowing for rapid deployment of new "stories" and innovation modules.
Media & Video Integration: Implemented performant video backgrounds and interactive video popups to enhance the site's storytelling without impacting page load speeds.
Clean UI Architecture: Focused on a high-density, whitespace-optimized layout to make complex scientific information digestible and professional.
Accessibility (a11y): Ensured the platform met WCAG 2.1 AA standards, focusing on semantic structure and inclusive navigation.
📈 Team Achievement
Successfully launched a tech-centric corporate platform that effectively positions Invaio as a leader in nature-positive agriculture.
Delivered a highly performant and accessible site that supports Invaio's global mission and partnership efforts.