Lead data Engineer

Qualifications:

1.   15+ years of experience in ETL & Data Warehousing

2.   Should have excellent leadership & communication skills

3.   Should have strong working experience on Data Lakehouse architecture

4.   Should have in depth knowledge on SSIS ETL Tool and good working knowledge on Power BI

5.   Should have worked on data sources such as SAP and Salesforce

6.   Should have very good knowledge of SSIS (ETL Tool), StreamSets (ETL Tool), Azure Cloud, ADF, Azure Synapse Analytics & Azure Hub Events

7.   Should have built solution automations in any of the above ETL tools

8.   Should have executed atleast 2 Azure Cloud Data Warehousing projects

9.   Should have worked atleast 2 projects using Agile/SAFe methodology

10. Should have demonstrated working knowledge on ITIL V4 concepts such as Incident Management, Problem Management, Change Management & Knowledge Management

11. Should have working experience on any DevOps tools like GitHub, Jenkins, etc & on semi-structured data formats like JSON, Parquet and/or XML files & written complex SQL queries for data analysis and extraction

12. Should have in depth understanding on Data Warehousing, Data Analysis, Data Profiling, Data Quality & Data Mapping

13. Should have cross global location experience and been part of a team with atleast 20+ members in a global delivery model

14. Should have experience in working with product managers, project managers, business users, applications development team members, DBA teams and Data Governance team on a daily basis to analyze requirements, design, development and deployment technical solutions

Responsibilities:

1.   Analyze the different source systems, profile data, understand, document & fix Data Quality issues

2.   Gather requirements and business process knowledge in order to transform the data in a way that is geared towards the needs of end users

3.   Write complex SQLs to extract & format source data for ETL/data pipeline

4.   Create design documents, Source to Target Mapping documents and any supporting documents needed for deployment/migration

5.   Design, Develop and Test ETL/Data pipelines

6.   Design & build metadata-based frameworks needs for data pipelines

7.   Write Unit Test cases, execute Unit Testing and document Unit Test results

8.   Deploy ETL/Data pipelines

9.   Use DevOps tools to version, push/pull code and deploy across environments