The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
I build scrapers that don't break when the target fights back.
Most scraping projects die the same way: script works locally, gets blocked in production, and suddenly you're burning through proxies with nothing to show for it. I've been there. So I built an engine that handles the anti-bot layer automatically.
The approach is simple: a fast light tier for normal sites, and a stealth browser tier that kicks in when Cloudflare, DataDome, or PerimeterX shows up. No manual switching. No CAPTCHA farms. The engine detects the challenge, escalates, clears it, and vaults the session so the next request doesn't have to start over.
Proof it works — tested live on my own hardware:
Google Search results: 33KB extracted via stealth browser, zero blocks
Hacker News: 34KB pulled in under 6 seconds on the light tier
Cloudflare's own diagnostic endpoint: 200 OK, challenge bypassed
The system is containerized, CI-tested, and built to run headless in production. I use it for my own data pipelines and I'm now taking on select client projects.
Typical engagements:
Competitor price monitoring across e-commerce catalogs
Real-time job board or real estate listing aggregation
SEO rank tracking at scale
Any structured data feed behind a WAF
What I need from you: 3 target URLs and a note on what data you need extracted. I'll return a sample dataset within 24 hours — no cost, no commitment. If the quality checks out, we discuss scope and pricing.
Not currently taking Facebook, LinkedIn, or Instagram scraping projects.
Here is the link to the completed work: https://github.com/mercury-systems/distributed-stealth-scraper
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Trending
retrofuturism
Share your favorite designs from the past exploring the distant future. Or your own take on the aesthetic 🚀
halftone
Dots. Texture. Grain. Let's see your designs using the halftone technique.
Midjourney
Creatives are showing off what they're making with Midjourney.
colorgrade
Video editors sound off: How do you enhance the mood of your video in color grading?
Related posts
Built an end-to-end pipeline that discovers local businesses without a website, enriches their contact details, and drafts personalized outreach using an LLM. Discovery is idempotent, keyed on a stable source ID, so no business is ever contacted twice across repeated runs. Includes a human-approval gate before any message is sent and output sanitization to stop model-generated commentary from reaching recipients. Runs unattended on a schedule, standard library only, and no ongoing infrastructure cost.
Manual lead generation often takes hours of repetitive work for marketing agencies. To solve this operational bottleneck, I built a custom B2B Lead Generation Engine.
Instead of relying on manual searches to qualify local businesses, I developed a script that automates the entire prospecting pipeline at scale:
📍 Local Accuracy: Integrates cleanly with the Google Places API.
🕵️♂️ Live Qualification: Uses Headless Browsing (Playwright) to detect active Facebook Pixels (instantly separating Hot vs. Cold leads).
📧 Contact Parsing: Scans the DOM and extracts valid contact emails using advanced RegEx.
📊 CRM Ready: One-click CSV export ready to be uploaded to HubSpot or GoHighLevel.
Built entirely with Python, Playwright, and Flask.
If your agency is looking to streamline its outreach and integrate a custom automated data pipeline, let’s connect. Send me a message and let's discuss your infrastructure needs.
#WebScraping #DataEngineering #B2BLeadGeneration #Python #Playwright #Automation #MarketingAgencies #TechConsulting #ProcessOptimization
Building an offline-first SaaS for cattle ranches:
One of the most interesting things I've built recently is BoviNest, a SaaS platform designed to help cattle ranches manage their animals, health records, traceability and daily operations.
But there's a particular constraint that makes the engineering considerably more interesting:
The application can't assume that there's an internet connection.
Cattle ranches aren't exactly known for having perfect connectivity.
So instead of treating offline mode as an edge case, I designed the mobile application around an offline-first architecture.
The app needs to be able to:
- Continue working when there's no connection
- Store changes locally
- Keep field operations uninterrupted
- Synchronize data when connectivity returns
- Handle failed requests and inconsistent states
- Work with RFID readers in the field
- Maintain consistency with the backend
The interesting part isn't simply making an app "work offline."
It's making the system behave predictably when connectivity is unreliable.
That requires thinking about the entire data flow:
Flutter → Local State → Persistence → Sync → API → PostgreSQL
rather than only thinking about the UI.
BoviNest is still evolving, but building it has taught me something I keep seeing across software projects:
Real-world constraints are where architecture starts to matter.
And that's one of the parts of software engineering I enjoy the most.
BoviNest: https://gitbonnest.com/bovinest
