Workforce Data ETL & KPI Pipeline (PostgreSQL, Python)
Designed a PostgreSQL + Python ETL pipeline processing workforce data into automated KPI
summaries, with SCD2 history tracking and MTD/YTD reporting.
PL/pgSQL stored procedures for KPI calculation and history tracking
Performance tuning: UNLOGGED staging tables, indexing, CTE-based transforms, bulk COPY
Automated daily CSV ingestion via scheduled Python/Pandas jobs
Stack: PostgreSQL, Python, Pandas, PL/pgSQL
0
2
PySpark Pipeline for Industry & Market Data Analysis
Built an end-to-end PySpark pipeline processing large-scale industry datasets to surface
business insights — top-paying roles, company growth trends, and financial metrics
(P/E ratio, ROE, headcount) across a 3-year window.
Designed Spark transformations for large dataset processing
Stored outputs in ORC/Parquet for efficient downstream querying
Exported results to MySQL via Sqoop for reporting
Stack: PySpark, Hadoop, ORC/Parquet, MySQL, Sqoop
0
3
Title: End-to-End API Data Pipeline (Python, Airflow, PostgreSQL, Docker)
Built a production-style ETL pipeline that pulls data from a REST API, transforms it, and
loads it into PostgreSQL on a schedule — fully containerized for portability.
Orchestrated with Apache Airflow (DAG-based scheduling, retries, alerting)
Implemented ELT transformations and automated data quality checks
Containerized with Docker for one-command deployment
Set up CI/CD so pipeline changes are tested before deploying
Stack: Python, Airflow, PostgreSQL, Docker, Git