Scrape Baby Products

Utilized Python's Scrapy library to extract valuable insights about the essential baby commodity (diapers). To make sense of this mountain of data, Prefect was employed for orchestration, ensuring a smooth and efficient data flow. The data was stored locally, mimicking cloud storage solutions like S3, Azure Bucket, or GCS. Eventually, all this data was integrated into a PostgreSQL database running on a personal computer, Docker was used to containerize all the ETL and Scrape applications.