Automated Web Scraper Project

A client needed a fully automated scraper to collect all business listings from BizBuySell.com — a popular marketplace for buying and selling businesses — and push the data into Airtable for analysis.

💡 Goal

Automate the entire data collection workflow — from fetching listings to parsing detailed info and updating Airtable — while staying undetected by modern Web Application Firewalls.

⚙️ What I Built

Asynchronous scraper using Playwright + BeautifulSoup, capable of scraping hundreds of pages efficiently.

A custom proxy integration with DataImpulse to automatically connect via a U.S. IP before every session — helping bypass anti-bot detection.

A modular architecture with separate layers for scraping, data parsing, proxy management, and Airtable integration.

Smart upsert logic for Airtable:

Daily scheduler that runs automatically at a fixed time using schedule — no manual trigger required.

🧩 Tech Stack

Python · Playwright · BeautifulSoup · Airtable API · Redis (optional) · DataImpulse Proxy · AsyncIO

🧠 Result

The scraper now runs fully unattended, collecting thousands of listings daily and keeping Airtable up to date — saving the client hours of manual work each day.

A video showing this process can be viewed here:

And the github repo for the project can be found here:

https://github.com/Ikhsan121/bizbuysell_scraper