Mandatory Skills: Spark, Scala and Python.
Technical & Other Skills
• 7+ years of working experience in scalable data pipeline using Scala,Python, pyspark, Hadoop, Apache Spark, Spark SQL, Kafka, Nifi, ETL and Incremental Data Load with Big data technologies.
• Experience of working with Databases like Oracle, Netezza and have strong SQL knowledge
• Developing scalable streaming solutions based on Hadoop/ Big Data stack such as HDFS, Ozone, Hive/ Impala, Apache Spark and Kafka.
• Experience in Data Engineering and implementing multiple end-to-end DW projects in Big Data environment
• Experience of building data pipelines through Git, Shell, Spark with Java/Python on Hadoop or Object storage (S3/CEPH)
• Strong analytical skills required for debugging production issues, providing root cause and implementing mitigation plan
• Strong communication skills - both verbal and written – and strong relationship, collaboration skills and organizational skills
·Flexible enough to stretch and work in India time to coordinate with India team
Experiences
• Nice to have development experience in Python based AI/ML workloads on Hadoop.
• Experience of working on Nifi, APIs and Unix Environment will be an added advantage
• Experience of working in Agile teams
• Ability to multi-task across multiple projects, interface with external / internal resources and provide technical leadership to junior team members
• Ability to be high-energy, detail-oriented, proactive and able to function under pressure in an independent environment along with a high degree of initiative and self-motivation to drive results
• Ability to quickly learn and implement new technologies, and perform POC to explore best solution for the problem statement
• Contribute ideas to help ensure that required standards and processes are in place and actively look for opportunities to enhance standards and improve process efficiency
• Perform assigned tasks and production incident independently
• Proven track record of delivering and willingness to roll up sleeves to get the job done.
Any Graduate