AI-Powered Website Research & Personalized

AI-Powered Website Research & Personalized Email Outreach Automation

Project Overview An end-to-end automated workflow built in n8n that combines deep website scraping, LLM-based content evaluation, RAG (Retrieval-Augmented Generation), and dynamic outreach generation. The system automatically ingests target domain links, processes sitewide content into concise context, and crafts personalized outreach emails sent directly through integrated messaging channels.

System Architecture & Core Modules

1. Input Trigger & Request Management

Multi-Channel Triggers: Ingests target URLs and outreach requests via automated schedules, database logs, and Telegram messaging triggers.

Rate Limiting & Safety Controls: Validates user/ID send quotas before execution to prevent spam flags, routing excess requests to fallback notification alerts.

Database Maintenance: Automated background triggers clean and purge historical execution logs periodically to optimize database performance.

2. Sublink Extraction & Web Scraping

Sitemap & URL Parsing: Executes HTTP requests across sitemap.xml and sitemap_index.xml endpoints to construct complete target URL hierarchies.

Dynamic Web Crawling: Utilizes crawl4ai integration alongside custom HTTP fetchers to extract clean raw content from live web pages.

Link Ranking: Evaluates scraped links using OpenAI LLM nodes to filter out low-value pages and prioritize high-intent sublinks for deep processing.

3. Content Processing & Page Summarization

HTML-to-Markdown Transformation: Converts raw page HTML into lightweight Markdown to eliminate boilerplate code and optimize token limits.

Data Cleansing & Aggregation: Trims content blocks to token budgets, flattens structured payloads, and aggregates multi-page data per domain.

LLM Summarization: Runs aggregated page text through OpenAI models to generate concise summaries capturing target business offerings, value props, and messaging hooks.

4. RAG Agent & Knowledge Base Querying

Vector Search Integration: Connects directly to a Postgres Vector Store (PGVector) via OpenAI Embeddings to retrieve contextual agency knowledge and baseline templates.

Instructor Agent: Employs a dedicated RAG AI Agent with persistent conversation memory to answer queries, fetch internal references, and align outreach messaging with proven offer frameworks.

5. Automated Email Generation & Outreach Execution

Contextual Copywriting: Generates tailored pitch emails using aggregated site summaries and retrieved RAG context.

Signature & Delivery: Standardizes signatures and sender details before executing native email dispatch.

Logging & Notifications: Writes sent email metadata back to centralized tables and broadcasts execution status directly via Telegram notifications.

Key Tools & Technologies Used

Workflow Automation: n8n

AI & Machine Learning: OpenAI GPT Models, Vector Store Embeddings, RAG Architecture

Vector & Relational Databases: PostgreSQL / PGVector, Custom Database Log Tables

Web Scraping & Extraction: Crawl4ai API, Dynamic XML Parsers, HTML-to-Markdown Processors

Communication APIs: Telegram Bot API, SMTP / Native Email Integrations