Data Architect

Job Description

 

• Experience working in Azure Databricks, Apache Spark
• Proficient programming in Scala/Python/Java
• Design and document data engineering and analytics solutions that follows design principles and modern data architectures
• Expert in developing and deploying data pipelines for processing data from streaming and batch data sources
• Experience setting up databricks environment in Azure, spark clusters, scalable and sustainable data engineering jobs
• Experience designing data flows, building data strategy and documenting the process
• Working knowledge in Azure Datalake, Azure Data Factory, Azure Log Analytics, Delta Lake, Kafka, Structured Streaming, DataFrame API, SQL, NoSQL Database
• Comfortable working in Azure services like Azure Key Vault, Azure Functions, Azure VNET
• Experience designing and developing database tables and/or schemas using SQL/No-SQL;
• Experience monitoring and maintaining the data pipelines and jobs
• Experience setting up data zones in delta lake and implementing fine grained access control
• Experience assessing and defining a backup, archiving, and disaster recovery strategy based on needs and costs
• Experience guiding other members of the team and ensure compliance with the architecture through team oversight
• Experience working in Power BI to setup data flows, create DAX queries, reports and dashboards with near real time updates
• Comfortable with API, Git, Notebooks, Spark Jobs, Performance tuning, Container-based deployments, Terraform
• Familiarity with Data Streaming Architecture, Azure Synapse, Azure NSG, data catalog, Azure Purview a plus.