Utilized Python's Scrapy library to extract valuable insights about the essential baby commodity (diapers). To make sense of this mountain of data, Prefect was employed for orchestration, ensuring a smooth and efficient data flow. The data was stored locally, mimicking cloud storage solutions like S3, Azure Bucket, or GCS. Eventually, all this data was integrated into a PostgreSQL database running on a personal computer, Docker was used to containerize all the ETL and Scrape applications.
0 likes · 3 views · Jul 21, 2024
Scrape data from the web related to baby products
