Responsibilities: • Analyzed data using Hadoop components Hive and Pig Queries, HBase queries. • Load and transform large sets of structured, semi structured, and unstructured data using Hadoop/Big Data concepts. • Involved in loading data from the UNIX file system to HDFS. • Responsible for creating Hive tables, loading data, and writing hive queries.

• Handled importing data from various data sources, performed transformations using Hive, Map Reduce/Apache Spark, and loaded data into HDFS. • Extracted the data from Oracle Database into HDFS using the Sqoop. • Loaded data from Web servers and Teradata using Sqoop, Spark Streaming API. • Utilized Spark Streaming API to stream data from various sources. Optimized existing Scala code and improved the cluster performance. • Experience in working with Spark applications like batch interval time, level of parallelism, memory tuning to improve the processing time and efficiency.

• Tech Stack: Hadoop 2.x · HDFS · spark SQL · Eclipse · Apache Kafka · Apache Sqoop · Apache Spark · Hive · Linux