Investing Platform Conversion by serhan sari

Investing Platform Conversion

serhan sari

Automation Engineer

Backend Engineer

Data Engineer

AWS Lambda

pandas

Python

● Responsible for converting 1.2 million Capital One Investing customers’ sensitive information to E*TRADE.

● Developed validation scripts with Python and Pandas for millions of rows of Capital One’s data

● Developed a time-saving method utilizing Pandas data frame with Python to find discrepancies in data migration during ETL jobs from the Sybase database.

● Used PySpark for real-time processing

● Developed SQL queries to extract data from existing sources and transferred the accounts and related data to E*TRADE.

● Involved in processing the streaming data and batch data using Apache Spark

● Imported information from different sources such as AWS S3, and EC2, and converted them into Spark RDD for analyzing data.

● Developed spark scripts via Pyspark by adhering to the client’s requirements for ensuring the effective operation

Like this project

Posted Nov 25, 2023

Converted 1.2M Capital One Investing customer data to E*TRADE, processed real-time data and ensured effective operation adhering to client requirements.

Likes

Views

Clients