Data Engineer

Data Engineer – Bigdata, Hadoop, Spark, Scala, PySpark, Strong SQL, Cloudera Data Platform (CDP), 
• Primary Skills:o Big Data: Spark, Scala, Pyspark, HDFSo Microsoft Stack: MS-SQL with strong knowledge in RDBMS conceptso Hadoop (Hortonworks) and Hiveo Cloudera Data platform (CDP)o Scripting Languages: Batch Script, Shell Script, Python
• Additional Skills:o Agile, Scrum, Jira, Git, SVN, Liquibase• Work on Pyspark Framework to ingest data from different sources system to Hadoop and SQL Server regions.
• Sound knowledge in Python and Good to have enough knowledge in Scala.
• Strong experience in database migration to MS SQL Server/Hadoop
• Extensive experience in database query tuning, performance tuning, and troubleshooting application issues on OLTP/OLAP systems.
• RDBMS Architecture, T-SQL query and Query Optimization knowledge and work experience
• Provide support to team members and helping them to understand the projects and requirements and guiding them to create the optimized solution of it.
• team player and proven track record of working in various team sizes performing cross-functional roles.
• Setup CICD pipeline for database changes using GitHub, Jenkin & Liquibase)
• Good to have experience data migration from Hadoop to CDP