Investing Platform Conversion

● Responsible for converting 1.2 million Capital One Investing customers’ sensitive information to E*TRADE.

● Developed validation scripts with Python and Pandas for millions of rows of Capital One’s data

● Developed a time-saving method utilizing Pandas data frame with Python to find discrepancies in data migration during ETL jobs from the Sybase database.

● Used PySpark for real-time processing

● Developed SQL queries to extract data from existing sources and transferred the accounts and related data to E*TRADE.

● Involved in processing the streaming data and batch data using Apache Spark

● Imported information from different sources such as AWS S3, and EC2, and converted them into Spark RDD for analyzing data.

● Developed spark scripts via Pyspark by adhering to the client’s requirements for ensuring the effective operation