Azure Databrick Architect

Role Specific Responsibilities Include: 

• 10+ years of experience in implementing large data and analytics platforms.
• 5+ years of experience architecting and building data platforms on Azure.
• 5+ years of experience architecting and building data platforms on Databricks from scratch.
• Pro in all Databricks engineering services Unity Catalog, Spark Jobs, Delta Live Table (DLT), DLT Meta, Databricks Workflow, Auto Loader. (You need to ask the candidate)
• Hands on in Databricks SQL, PySpark
• Understanding of data modelling, data mappings, understand the data model changes and propose the best practices and guidelines.
• Creating technical design based on end-to-end data architecture and frameworks for meta data driven data ingestion, transformation, logging and monitoring.
• Deploying data pipelines/applications to higher environments using CI-CD pipelines
• Experience in writing technical user stories.
• Experience in Agile methodology and scrums.

Communication:

• Excellent communication, presentation and client handling skills written and verbal. Should be able to drive technical discussions with client & partners and share technical thought leadership in architecture and design discussions.

Key responsibilities:

• Design Azure Databricks platform architecture from scratch.
• Design and build ingestion framework, data processing framework, logging & monitoring framework and, data sharing framework.
• Share Databricks best practices.
• Develop data models in different layers of Databricks.
• Design, build and orchestrate data pipelines in Azure and Databricks
• Define strategy for configuration management and CI/CD
• Perform POCs