Data Engineer

Job Description:-

·         Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.

·         Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.

·         Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

·         Strong analytic skills related to working with unstructured datasets.

·         Build processes supporting data transformation, data structures, metadata, dependency and workload management.

·         A successful history of manipulating, processing and extracting value from large, disconnected datasets.

·         Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.

·         Strong project management and organizational skills.

Mandatory Skills:

·         Experience with big data tools: Hadoop, Spark, Kafka, etc.

·         Experience with AWS cloud services: EMR, redshift, athena, sql, etc.

·         Should have good experience in BI tools like ETL, informatica

·         Should have worked for CI/CD tools

·         Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.

·         Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.

·         Experience with stream-processing systems: Storm, Spark-Streaming, etc.