Project Description:
The Project
is about to handle Risk Management Team, where the Bank wanted to store,
process manage the huge amount of data in a day-to-day operation, collected
from various sources. The system Majority checks the credibility of the
customer looks for the credit risks.
Responsibilities:
·
Ingested data from multiple sources like MySQL.
·
Created and worked on Sqoop jobs with incremental load.
·
Design both managed & External tables in Hive.
·
Developed Spark Code in Scala using Spark SQL & Data frames for
optimization.
·
Creating HBase layer for faster reporting.
· TechStack: HDFS · Apache Sqoop · HBase · Apache Spark · Hive · Hadoop
