The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
I build scrapers that don't break when the target fights back.
Most scraping projects die the same way: script works locally, gets blocked in production, and suddenly you're burning through proxies with nothing to show for it. I've been there. So I built an engine that handles the anti-bot layer automatically.
The approach is simple: a fast light tier for normal sites, and a stealth browser tier that kicks in when Cloudflare, DataDome, or PerimeterX shows up. No manual switching. No CAPTCHA farms. The engine detects the challenge, escalates, clears it, and vaults the session so the next request doesn't have to start over.
Proof it works — tested live on my own hardware:
Google Search results: 33KB extracted via stealth browser, zero blocks
Hacker News: 34KB pulled in under 6 seconds on the light tier
Cloudflare's own diagnostic endpoint: 200 OK, challenge bypassed
The system is containerized, CI-tested, and built to run headless in production. I use it for my own data pipelines and I'm now taking on select client projects.
Typical engagements:
Competitor price monitoring across e-commerce catalogs
Real-time job board or real estate listing aggregation
SEO rank tracking at scale
Any structured data feed behind a WAF
What I need from you: 3 target URLs and a note on what data you need extracted. I'll return a sample dataset within 24 hours — no cost, no commitment. If the quality checks out, we discuss scope and pricing.
Not currently taking Facebook, LinkedIn, or Instagram scraping projects.
Here is the link to the completed work: https://github.com/mercury-systems/distributed-stealth-scraper
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Related posts
Malware in a WordPress backup, disguised as a photo
A client wanted me to restore a WordPress site from a zip backup and asked for one thing before paying: a screenshot of the site running on my own computer, with localhost in the address bar.
Loading a WordPress site runs its PHP on whatever machine serves it. So I read the zip with a script first and didn't extract it.
WordPress core matched the official checksums. In wp-content/mu-plugins, though, there was a file that looked for a 4.6 MB "JPEG" in uploads, renamed it to .exe and tried nine ways to launch it. The first two bytes of that "JPEG" were MZ, which is how a Windows executable starts. Opening localhost for the screenshot would have been enough to run it.
My first script missed it because it trusted file extensions. Now it reads the first bytes of every file.
The write-up has the loader code and the checks I now run before importing a backup I didn't make. If the same file reached you, the SHA-256 hashes are at the end.
https://helban.dev/en/case-studies/fake-wordpress-backup-malware/
The MZ check is such a good catch. Worth adding for anyone restoring sites: mu-plugins run on every request and can't be switched off from the dashboard, so they're a favourite hiding spot. Did the client ever reply after you flagged it?
Building an SEO SaaS? You might not need another expensive keyword API subscription.
A question for indie hackers and developers:
How do you currently access keyword data for your applications?
If you're building programmatic SEO pages, content research automation, keyword discovery tools, or an AI-powered marketing product, you probably need structured search data.
We developed the SerpSpur Universal REST API to make this workflow more flexible.
The API provides access to:
Search volume and historical trends.
Keyword difficulty and backlink competition.
CPC and advertising competition.
Search intent and parent topics.
Related keyword clusters.
SERP features and forum weakness signals.
AI Overview-related information.
Bulk keyword enrichment.
It also supports different research depths, from a single keyword to larger semantic clusters.
One feature worth exploring is the 30-day cache Vault mode: cached lookups cost credits, while cache misses return without a credit deduction.
Great work! I appreciate how clearly the design prioritizes usability without compromising its visual appeal.
Datalier — AI-Powered Data Enrichment
Incomplete records make it harder to find the right people. Datalier turns scattered account data into clear insights, helping go to market teams focus on the right conversations.
Nice way to surface account gaps.