Description

• ·      Very Strong engineering skills. Should have an analytical approach and have good programming skills.
• ·      Provide business insights, while leveraging internal tools and systems, databases and industry data
• ·       Minimum of 5+ years’ experience. Experience in retail business will be a plus.
• ·       Excellent written and verbal communication skills for varied audiences on engineering subject matter
• ·      Ability to document requirements, data lineage, subject matter in both business and technical terminology.
• ·      Guide and learn from other team members.
• ·      Demonstrated ability to transform business requirements to code, specific analytical reports and tools
• ·     This role will involve coding, analytical modeling, root cause analysis, investigation, debugging, testing and collaboration with the business partners, product managers other engineering team.

Key Responsibilities:

·         Development experience with Java, Scala, Flume, Python

·         Development background with Spark or MapReduce/YARN is a must have

·         This candidate is expected to have significant application development experience in the past either with Java or Python. We are looking for someone who was a Software Engineer in the past and turned into a Data Engineer and build end to end data pipelines from ground up

·         Knowledge/experience on Teradata Physical Design and Implementation, Teradata SQL Performance Optimization

·         Advanced SQL (preferably Teradata)

·         Experience working with large data sets, experience working with distributed computing (MapReduce, Hadoop, Hive, Pig, Apache Spark, etc.).

·         Strong Hadoop scripting skills to process petabytes of data

·         Experience in Unix/Linux shell scripting or similar programming/scripting knowledge

·         Experience in ETL/ processes

·         Real time data ingestion (Kafka)