Yellow Pages Scraper for Market Research & Lead Generation

Eren FALAY

Data Scraper

Lead Generator

Market Researcher

pandas

Playwright

Python

Data

Project Description

Extracted detailed company data from the Gelbe Seiten (German Yellow Pages) directory based on user-defined keywords and locations, then saved it into structured CSV files for further analysis.

Project Overview

In this project, I developed a scalable, high-performance web scraper for Gelbe Seiten, Germany's largest business directory. Users can specify keywords, locations (e.g., “dentist”, “Berlin”), and the scraper retrieves a detailed list of businesses that match these criteria. The script efficiently handles pagination, extracts detailed company data (name, address, contact info, ratings, etc.), and exports it into a clean, well-structured CSV files. Optimized for speed and resilience, the scraper manages concurrent requests and retries failed ones to ensure data integrity. This tool is ideal for market analysis, lead generation, and competitive intelligence.

Project Breakdown

Parsed dynamic search results from AJAX endpoints to extract unique company IDs for detailed data retrieval.

Extracted key information: business names, categories, ratings, addresses, phone numbers, emails, websites, and more.

Optimized performance and reliability through a few key strategies:

Asynchronous design allowed for concurrent HTTP requests, significantly reducing scraping time.

A resilient retry strategy was designed to automatically handle common issues like timeouts and server errors (e.g., 429, 500).

Data was processed in chunks to optimize memory usage.

A flexible CLI allows users to perform multi-keyword and multi-location queries.

Results were stored in separate CSV files, organized by keyword and location.

Outcome & Insights

The GelbeSeiten Business Data Scraper produces clean, structured datasets for a variety of business applications. By providing a streamlined method for extracting up-to-date contact and company information, the tool enables businesses, researchers, and market analysts to perform competitive analysis, generate targeted leads, and conduct detailed regional market studies with minimal effort.

Real-Life Scenario

A client in the dental industry wanted to gather a list of dentists in Berlin to target with a new product offering. Using this scraper, they were able to pull detailed contact information for over 1000 dentists in the city, complete with ratings, phone numbers, and websites. This provided a highly targeted marketing list in just a few hours.