Data Engineer

Job Description
Responsibilities:
·        Develop and integrate software applications using suitable development hashtag#methodologies and standards, applying standard hashtag#architectural hashtag#patterns.
·        Build hashtag#libraries, user defined functions, and hashtag#frameworks around hashtag#Hadoop
·        Develop user defined functions to provide custom hashtag#hive, hashtag#HDFS, hashtag#Kafka and hashtag#SPARK capabilities
·        Develop and execute test scripts independently or in concert with testing personnel.
·        Collaborate with hashtag#BusinessAnalysts, hashtag#Architects and Senior Developers to establish the physical application hashtag#framework (e.g. hashtag#libraries, hashtag#modules, hashtag#execution environments).
·        Perform end to end automation of hashtag#ETL process for various hashtag#datasets that are being ingested into the hashtag#bigdata platform.
·        Follow best practices and hashtag#develop code as per standard defined
Required Tools/Languages:
Must Have
· hashtag#CoreJava, hashtag#Spark or hashtag#Scala
·        Strong hashtag#SQL
· hashtag#Unix / hashtag#Linux hashtag#ShellScripting
·        Strong understanding of hashtag#Hadoop internals
· hashtag#HIVE/ hashtag#Impala
· hashtag#HBase
· hashtag#Python
·        Knowledge on hashtag#cloud is plus.