Data Engineer

As Data engineer,

• • Develop and build AI/ML model ready data ponds using Azure Databricks Delta lake
• • Improve upon the data ingestion models, ETL jobs, and alarm to maintain data integrity and data availability.
• • Partner with AI/ML product teams proactively identify the data needs for rapid experimentation, stable production deployment and ensure the required data availability by collaborating with Product teams.
• • Leverage domain knowledge, suggest optimal path to secure the data required for AI/ML model experimentation and future ML Ops
• • Write complex and efficient queries to transform conventional data sources into easily accessible models
• • Knowledge of data management fundamentals and data storage principles
• • Continuously explore the advances big data technologies and advocate the related innovations across the digital organization

Data Engineer 
The AI/ML CoE is looking for Data Engineer with ETL, MDM background and ERP (Sales, Inventory, AP and AR) (or) Industrial manufacturing domain knowledge (or) chemical lab information management systems that enables him/her to be proactive and add value to related AI/ML product development. Accountable to solve the problems that require extensive use of ETL and MDM techniques 
As Data engineer, 
• Develop and build AI/ML model ready data ponds using Azure Databricks Delta lake 
• Improve upon the data ingestion models, ETL jobs, and alarm to maintain data integrity and data availability. 
• Partner with AI/ML product teams proactively identify the data needs for rapid experimentation, stable production deployment and ensure the required data availability by collaborating with Product teams. 
• Leverage domain knowledge, suggest optimal path to secure the data required for AI/ML model experimentation and future ML Ops 
• Write complex and efficient queries to transform conventional data sources into easily accessible models 
• Knowledge of data management fundamentals and data storage principles 
• Continuously explore the advances big data technologies and advocate the related innovations across the digital organization 

Required Qualifications 
•7 + Years’ hands on experience in data engineering 
•Experience working with Databricks and Apache Spark/PySpark 
••Experience with cloud-based data services, including data pipeline orchestration tooling (i.e. Azure Data Factory). 
•Proficiency with complex SQL development 
•Experience in modern DevOps practices (including Git, CI/CD) 
•Experience in Data Modeling. 
•Strong business acumen and adaptability to partner with the AI/ML product teams on innovative solutions to constantly changing business requirements 
•Any ETL experience in design, mapping and configuration in a complex environment processing large