Python Pipeline for Aggregating US Municipal Building PermitsPython Pipeline for Aggregating US Municipal Building Permits
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Aggregating US municipal building permits across different counties into a unified lead feed using Python automation pipelines.
The real bottleneck in municipal scraping isn’t parsing HTML — it’s the fragmentation of municipal portal architectures (legacy ASP.NET, dynamic GIS viewers, and session-locked forms).
How we structured the pipeline:
• Asynchronous crawling pipelines with request queuing
• Normalized schema parser converting raw permit tables into structured JSON
• Automated deduplication engine running on daily trigger workflows
Live demo tool:
Anyone else building custom scrapers for Gov / municipal portals? How do you deal with legacy ASP.NET postbacks?

asp.net

ASP.NET Core | Open-source web framework for .NET

Build web apps and services that run on Windows, Linux, and macOS using C#, HTML, CSS, and JavaScript. Get started for free on Windows, Linux, or macOS.

Back to feed
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started