Data Engineer

Responsibilities

·Work as part of a team to develop Cloud Data and Analytics solutions

·Participate in development of cloud data warehouses, data as a service, business intelligence solutions

·Ability to provide solutions that are forward-thinking in data and analytics

·Deliver a quality product

·Coding complex Spark (Python), T-SQL

·Developing Modern Data Warehouse solutions using Azure Stack (Azure Data Lake, Azure Data Factory, Azure Databricks, Azure Synapse)

Skills/ Qualifications:

·Hands-on experience in Azure Databricks, Azure Data Factory, Azure Synapse – Mandatory

·Good understanding of Azure Databricks platform and ability to build data analytics solutions to support the required performance & scale

·Working with Spark cluster and effective usage of spark engine for any data transformation creating notebooks with PySql scripts

·Ability to Read and Write data to Azure Data Lake or Blob storage from Databricks

·Good understanding and experience working on Delta Lake using Apache Delta in Azure Databricks building Data Pipelines for File, Database and API based ingestion using different activities within and also with Data flows for any Data Transformation

·Good Knowledge on Global parameters, Triggers, Linked services, Datasets and Arm template

·Experience of creating tables, building dimensional modelling

·Experience as Data Engineer in Azure Big Data Environment

·Expertise in ETL tools i.e. (SSIS)

·Expertise in Implementing Data Warehousing Solutions

·Experience with working in Agile (KANBAN, Scrumban) environment as part of a scrum team

·Programming experience in Python, T-SQL

·Hands-on experience in Azure stack, with good knowledge on Azure Key Vault, Active directory and basic understanding of Devops and SSIS

·Experienced working on performance tuning and optimizing long running jobs

·Demonstrated analytical and problem-solving skills, particularly those that apply to a big data environment

·Good understanding of Modern Data Warehouse and Data Warehousing concepts

·Proficient in a source code control system (e.g. ADO,gitHub)