The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
Most scraping projects don't fail because of anti-bot walls. They fail because of bad error handling.
Built this as a portfolio demo, same logic scales to any catalog or listing site.
Retries failed requests with exponential backoff instead of giving up after one bad response. Random 0.5-1.5s delay between pages instead of hammering the server on a fixed timer. Follows pagination automatically until the last page. Exports straight to formatted Excel: headers, filters, frozen row, ready to hand off.
Full pipeline and the actual retry/parse code in the images.
What's the ugliest edge case that's broken a scraper on you? Timeouts, layout changes, or CAPTCHAs?
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Related posts
Noise Reveal, a premium reveal component for Framer
I built a component that gives Framer sites cinematic reveals for images, video and live layouts. It has a soft, film-like edge with glow and grain, and can play on load, on view, from a variant or scrubbed by scroll.
It’s built for production: the reveal is prepared before it scrolls in, it’s accessible with reduced-motion support, it falls back when WebGL isn’t available, and pages stay plain HTML when nothing is moving.
Launched on the Framer Marketplace. Open to custom Framer and interaction work.
The WebGL fallback and reduced-motion support are the parts most reveal effects skip, so it's great to see them called out. Does the fallback keep some of the grain edge with CSS, or does it drop to a simple fade?
Zencastr came to us with a complex mix: they hosted their homepage on a custom server, blog on a separate platform and the rest of the pages on Webflow.
They had 3 days before the launch to tie it all together. This is their review:
Three platforms behind one domain in three days is a real test. I'd guess the reverse proxy and keeping the nav and footer identical across all three took more care than any of the page building. Did you match the shared header in Webflow or pull it from one source?
Malware in a WordPress backup, disguised as a photo
A client wanted me to restore a WordPress site from a zip backup and asked for one thing before paying: a screenshot of the site running on my own computer, with localhost in the address bar.
Loading a WordPress site runs its PHP on whatever machine serves it. So I read the zip with a script first and didn't extract it.
WordPress core matched the official checksums. In wp-content/mu-plugins, though, there was a file that looked for a 4.6 MB "JPEG" in uploads, renamed it to .exe and tried nine ways to launch it. The first two bytes of that "JPEG" were MZ, which is how a Windows executable starts. Opening localhost for the screenshot would have been enough to run it.
My first script missed it because it trusted file extensions. Now it reads the first bytes of every file.
The write-up has the loader code and the checks I now run before importing a backup I didn't make. If the same file reached you, the SHA-256 hashes are at the end.
https://helban.dev/en/case-studies/fake-wordpress-backup-malware/
The MZ check is such a good catch. Worth adding for anyone restoring sites: mu-plugins run on every request and can't be switched off from the dashboard, so they're a favourite hiding spot. Did the client ever reply after you flagged it?