Bandsintown Event Scraper Development

Role: Developer | Tech: Python, Playwright, JSON, Logging

The Bandsintown Event Scraper is a browser-automation tool I built to extract structured live-event data from the Bandsintown platform. It automates authentication, navigates multiple artist pages, and saves event details (date, venue, city, country) into clean JSON datasets. This project highlights my ability to design scalable, reliable, and production-grade automation pipelines for complex web platforms.

🔐 Automated Login – Simulates user authentication and stores cookies for persistent sessions.

🎫 Smart Event Extraction – Gathers complete event data across multiple artist pages.

🧹 Duplicate Filtering – Ensures clean and unique event records.

🧩 Batch Processing – Scrapes dozens of artists automatically from a database.

🪵 Detailed Logging – Provides rich logs for transparency and debugging.

📦 JSON Output – Structured, analysis-ready data export.

Layer Tools / Libraries Language Python 3.8+ Framework Playwright Data Handling JSON, asyncio Utilities logging, os Environment Cross-platform (Windows, macOS, Linux)

Input Preparation – Load artist data from artists_database.json.

Login Automation – Authenticate once; session cookies are stored.

Scraping Phase – Visit each artist’s event page and extract details.

Data Consolidation – Merge results using combine_json.py.

Output files are stored under /events/ and merged into a single dataset when needed.

Automated event collection for hundreds of artists.

Reduced manual research time by over 90%.

Produced clean, deduplicated JSON datasets ready for analytics or dashboards.

Designing resilient web scrapers for dynamic sites.

Implementing scalable batch automation.

Building maintainable systems with structured logs and clean architecture.

Delivering end-to-end automation solutions from concept to deployment.

Developed scraping logic and browser automation with Playwright.

Designed data structure and duplicate-filtering mechanism.

Created logging and JSON merging utilities.

Authored full technical documentation.

Open-sourced under the MIT License.

I’m M. Ikhsan Arif, a Data Automation Engineer & Physicist passionate about transforming unstructured web data into reliable insights. I build automation tools, data pipelines, and intelligent scrapers that make data collection faster, smarter, and more maintainable.