Problem:
Batch data processing was not suitable for real-time analytics and scalable cloud-based data ingestion.
Solution:
Created a real-time streaming pipeline using Kafka on AWS EC2, stored processed data in S3, cataloged it with AWS Glue, and queried it with Amazon Athena.
Tools:
Python, Apache Kafka, AWS EC2, Amazon S3, AWS Glue, Amazon Athena, Pandas
Result:
Built an end-to-end cloud data streaming workflow that supports real-time ingestion, storage, cataloging, and SQL-based analytics.
0 likes · 0 views · Jun 25, 2026
Problem: Batch data processing was not suitable for real-time analytics and scalable cloud-based data ingestion. Solution: Created a real-time streaming pi...
