Description

Job Description / Responsibilities:

  • Build data pipelines that migrate data from Hadoop to Snowflake.
  • Work closely with data analysts and business end-users to implement and support data platforms using best-of-breed technology and methodology.
  • Analyse systems operations data (SLAs, customer satisfaction, delivery quality, team efficiency etc.) to identify actionable trends for continual improvements.
  • Work closely with data platform teams to build systematic process and services to support efficient onboarding and management of data sets into the data virtualization layer.
  • Design and implement scalable data pipelines and data service modules with robustness in-mind to support the growing demands from business users.

Must have skills

  • Informatica BDM
  • Hadoop (HDFS, Hive, Impala)
  • SQL
  • Python
  • Shell Scripting

Requirements:

  • Bachelor Degree in Computer Science, Computer Engineering or equivalent.
  • At least 5 years’ experience of working as a data engineer in big data field.
  • Solid working knowledge of implementing ETL pipelines using Informatica BDM (DEI) on data warehouses and big data platforms, such as RDBMS, Hadoop, Snowflake.
  • Familiar with application integration with RDBMS such as Oracle, MS-SQL or MySQL.
  • Working knowledge of Oracle and MS-SQL will be a plus.
  • Hands-on experience of using Linux (or Unix-like OS) as the development environment and familiar with shell scripts and command line tools in Linux/Unix environment.
  • Exposure and knowledge in the following technologies is advantageous:
    • Big Data Platforms: Snowflake, Hadoop (Spark, Hive, Impala, HDFS)
    • Programming and Scripting: Python, Shell Script
    • SQL Databases: Oracle, MS-SQL
    • AWS Services: S3, EMR
  • Experienced with the Systems Development Life Cycle implementation methodology (SDLC) and/or agile methodologies like Scrum and Kanban.
  • Understand and apply the good industry practice of code versioning, testing, CICD workflow and code documentation.
  • Good team player, with strong analytical skills and enjoy complex problem solving with innovative ideas.
  • Strong communication/people skills required to interact with data analysts, business end-users and vendors to design and develop solutions.
  • Good at working with details and is meticulous for operations.

Good to have skills:

  • Working knowledge on Oracle and MS-SQL
  • Snowflake
  • AWS Services (S3)

Key Skills
Education

Any Graduate

Back To Jobs