Build Data Pipeline with Python on Cloud
Subrin Al Azad
5
Starting at
$
50
/hrAbout this service
Summary
If you want to build or customize your data pipeline, then this is the perfect gig for you. As an experienced Data Engineer, who has been working in this field for a very long time, I can build entire Data Pipelines involving ETL (Extract, Transform & Load) operations using Python and Integration with Cloud Functionalities.
What's included
I will develop the followings
I will do the following in this process:- 1. Extraction/Connect: - From any kind of websites including e-commerce, business sites - From login required site - From website having hidden APIs - Develop custom scrapers - Database integration with scrapers 2. Transformation: - Merging - Aggregating - Summarizing - Filtering - Enriching - Splitting - Joining - Removing duplicated data and many more ... 3. Loading into any storage over cloud or locally. - Locally: In any type of database (MySQL, MongoDB, Postgres, MariaDB, etc...) or in any flat-file like (JSON, CSV, TSV, etc...) - On Cloud: AWS S3, Google Cloud Storage, Azure Blob Storage, etc ... (any cloud service that you want)
Example projects
Recommendations
(5.0)
Client • Feb 21, 2023
Recommended
I found him to be professional and meticulous he is hard working, dedicated, capable I found his conduct to be well-behaved and courteous, he puts the need of the customer first he is an expert in data analysis, python programming and photo retouching
Skills and tools
Work with me