Data Engineering
Starting at
$
150
/hrAbout this service
Summary
I create modularized ETL automation & optimization framework that improves the infrastructure of data streams for various projects. The framework enables backend developers to pick relevant modules and effortlessly complete data streams among a variety of data systems in Teradata, SQL Server, Oracle, and SAP HANA.
I manage Linux VM servers, schedule cron jobs, automate data loads and transformations, utilize parallel processing, develop bots to scrape critical messages from log files and post to collaboration channels such as Git, Mattermost, or Slack.
What's included
ETL Framework
Linux shell script based framework that can orchestrate various ETL operations
ETL Automation
Full automated ETL that can extract data from and load data into various database platforms
Data Flow Diagrams
Clear Data Flow Diagrams showing a clear picture of how data streams are orchestrated
Data Dictionaries
Detailed data dictionary documenting data fields, data types, constraints, indexes, etc.
