Projects using AWS VPC in LahoreProjects using AWS VPC in Lahore
Cover image for Real-Time Analytics Platform on AWS
Real-Time Analytics Platform on AWS designed to ingest, process, and visualize high-volume streaming data from IoT devices and application logs. Ingestion: Device telemetry (IoT Core) and application logs are published into Amazon Kinesis Data Streams, followed by lightweight routing and enrichment using AWS Lambda. Processing & Storage: Amazon EMR performs heavy data transformations via Spark Structured Streaming, landing the curated output into Amazon S3 as partitioned Parquet files. AWS Glue then handles batch transformations before loading the data into Amazon Redshift for the analytics workload. Orchestration & Serving: AWS Step Functions coordinate the batch processing workflows. Finally, Amazon QuickSight generates actionable business dashboards while Amazon CloudWatch monitors operational metrics and alarms
0
515
Cover image for Batch ETL Data Warehouse Modernization
Led
Batch ETL Data Warehouse Modernization Led modernization of a legacy batch ETL pipeline into a scalable AWS-based data warehouse. Replaced manual SQL jobs with an automated system loading diverse sources into Amazon Redshift. Ingestion: Daily CSVs and API data land in S3; S3 events trigger Lambda to launch Glue Crawlers and update the Data Catalog. Transformation: Glue PySpark jobs join and cleanse sales, inventory, and customer data, apply quality checks, and move errors to a quarantine path. Output: Partitioned Parquet files stored in S3 for loading into Redshift.
0
300