Evolving a Legacy Scraping Script into a High-Performance B2B Lead Intelligence Engine ๐ I reali...Evolving a Legacy Scraping Script into a High-Performance B2B Lead Intelligence Engine ๐ I reali...
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Evolving a Legacy Scraping Script into a High-Performance B2B Lead Intelligence Engine ๐
I realized early on that traditional lead scrapers hit a wall very quickly. Basic HTTP requests frequently fail against anti-bot barriers, miss dynamic JavaScript tracking tags (like Meta Pixels loaded asynchronously), and leave sales teams with unverified, low-quality data. On top of that, standard AI wrappers often hallucinate company details, destroying cold outreach credibility.
To fix this, I completely refactored the pipeline into a full-stack, production-ready B2B Lead Intelligence Engine built with FastAPI, Playwright Async, and Gemini API.
Key Architectural & Product Highlights:
โก Dynamic Execution & Pixel Audit: Migrated from static HTTP requests to headless Playwright Async, dynamically evaluating window.fbq post-rendering to eliminate false negatives in Meta Pixel detection.
๐ก๏ธ Zero-Hallucination AI Enrichment: Powered by Gemini in Strict JSON Mode, the system extracts company summaries, target audiences, and personalized email icebreakers. Crucially, it enforces local evidence validationโevery AI claim must match verbatim text extracted from the prospect's site.
๐ฏ Human-in-the-Loop (HITL) Workflow: Built an intuitive dashboard allowing operators to review, edit, approve, or discard leads before exporting cleaned CSVs to CRMs (HubSpot, Lemlist, Instantly).
๐ Automated Lead Scoring: Features an algorithmic 0โ100 quality score (Grades A, B, and C) based on contact completeness, tech stack footprint, and AI confidence.
๐พ Resilient Async Backend: Integrated SQLite background job persistence with cooperative task cancellation and asynchronous batch processing.
By bridging deep web automation with strict AI governance, this project transforms raw directory listings into fully audited, high-converting outbound campaign assets in seconds.
Clients, be honest: what makes you hire a freelancer fastest?
A) A portfolio piece that looks just like my project
B) A short, specific proposal
C) A quick call where they get it right away
D) Reviews and a track record
I'll go first. For automation and scraping work, it's usually C. A 10-minute call where I ask the right questions about their data beats any proposal I could write.
Drop your letter below ๐ Freelancers, I'd love to hear which one actually wins you work.
C for me too and in my case the call does double duty. I build AI voice agents, so on a short call the client does not just hear me explain the work, they can hear the actual demo agent answer and book a test appointment live.
Ten minutes of hearing the real thing beats any...
I haven't posted a proper update here for a little while. I've been busy building a couple of AI applications and a healthcare workflow prototype, and writing about the work kept slipping down the list.
So, a small catch-up. :)
One project is an AI presentation builder for business teams. It starts with information about the business, its audience and the conversation they're preparing for, together with approved source material. From there, it helps create a presentation outline, a talk track, speaking notes and preparation notes.
The person using it can review the material, change it, save it and come back later to export it. That sounds fairly ordinary when written down, but those details are what make it something a team can actually work with. The connected pilot has been accepted, and I'm continuing to refine the content and develop the application.
Another project is a knowledge and planning workspace. Imagine having useful advice buried in hours of coaching transcripts. The application turns that material into a library of teachings that people can review, with links back to the sources.
It also supports a planning workflow: start with a client brief, confirm the details, then build a 90-day roadmap with check-ins and weekly updates. The knowledge foundation has been accepted; the planning workflow is on staging while the remaining integration work continues.
Alongside those, I've been working on a healthcare product prototype. This one is about showing how a complicated business process behaves: someone approves a decision, a member makes a choice, a referral moves forward, a booking happens, and someone needs help when the normal path doesn't fit.
It uses synthetic data and scripted scenarios, so the team can walk through the product without involving real patient records. Both demonstration releases were accepted, followed by a separate piece of work planning the next pilot.
I've kept the client names and interfaces private, but wanted to explain the work a little better than just saying "AI integrations" or listing frameworks.
These are the kinds of projects I'd like to do more of: useful AI features, B2B applications, and consumer products where the interface and the underlying workflow both get proper attention. I enjoy being involved in the whole thing, from figuring out what someone needs to do through to building the application that lets them do it.
I'm also updating my portfolio to make that work easier to find. You can see my public projects at https://nikocodes.com, including ReachLog and ThaiGlow.
If you're building something in this area, feel free to message me here. Happy to talk through what a sensible first version could look like.
Using synthetic data and scripted scenarios for the healthcare prototype is a smart call. Letting stakeholders walk the exception path, not just the happy path, is usually where a pilot earns its trust.