Data Engineer

Job Description

5+ years of Data Engineering experience

Mandatory Skills : - Hive, Spark, Scala, Python, Linux, Shell scripting

Experience in designing, implementation and maintenance of high-performance big data infrastructure, systems & processing pipelines scaling to billion records of structured and unstructured data.

Technical expertise with data models, data mining, and segmentation techniques

Must implement data pipelines to automate the ingestion, transformation, and augmentation of data sources, and provide best practices for pipeline operations

Hands-on experience using Hive, Spark, Scala, Python, Linux, Shell scripting.

Hands on experience with Spark Streaming or other live stream processing technologies would be a plus including building Flask and Rest APIs

Any Cloud development (AWS, Azure, GCP) would be a plus and added advantage

Oversee assigned programs and provide guidance to team members

Assist with solving technical problems when they arise

Explore new technologies and learn new techniques to solve business problems creatively.
Desired Skills and Experience
Spark, scala, python, hive, shell scripting, Linux, Apache Spark Streaming