Most scraping projects don't fail

Most scraping projects don't fail because of anti-bot walls. They fail because of bad error handling.

Built this as a portfolio demo, same logic scales to any catalog or listing site.

Retries failed requests with exponential backoff instead of giving up after one bad response. Random 0.5-1.5s delay between pages instead of hammering the server on a fixed timer. Follows pagination automatically until the last page. Exports straight to formatted Excel: headers, filters, frozen row, ready to hand off.

Full pipeline and the actual retry/parse code in the images.

What's the ugliest edge case that's broken a scraper on you? Timeouts, layout changes, or CAPTCHAs?