Azure Data Engineer 2

Job Description

Required:

4+ years of Python experience

Strong experience with SQL, querying with extensive join conditions and updating based on conditions.

DevOps: Linux, GitHub, Bash

Experience with CI/CD lifecycles, 

Experience writing Unit, integration, and regression tests.

Experience with static code analysis tools, Linters, and PEP 8 standards

Experience with Microsoft Azure and/or Google Cloud Platform

Experience with Agile SCRUM projects.

Highly Desired:

Data Management: Databricks, PySpark, Spark Structured Streaming, Delta Live Tables, Delta Sharing, 

Expert knowledge of SQL

Experience with Streaming Technologies: Apache Kafka, Azure EventHubs, Avro

DevOps: GitHub Actions, Terraform, Artifactory

Desired

Java, SpringBoot, Kubernetes, Docker, NoSQL

Position Summary

Accountable for developing and delivering technological responses to targeted business outcomes. Analyze, design and develop enterprise data and information architecture deliverables, focusing on data as an asset for the enterprise. Understand and follow reusable standards, design patterns, guidelines, and configurations to deliver valuable data and information across the enterprise, including direct collaboration with, where needed. Demonstrate the company's core values of respect, honesty, integrity, diversity, inclusion and safety.

Key Responsibilities

Key Responsibilities

• Utilize enterprise standards for data domains and data solutions, focusing on simplified integration and streamlined operational and analytical uses
• Ensure there is clarity between ongoing projects, escalating when necessary, including direct collaboration Leverage innovative new technologies and approaches to renovate, extend, and transform the existing core data assets, including SQL-based, NoSQL-based, and Cloud-based data platforms
• Define high-level migration plans to address the gaps between the current and future state
• Contribute to the development of cost/benefit analysis for leadership to shape sound architectural decisions
• Analyze technology environments to detect critical deficiencies and recommend solutions for improvement
• Promote the reuse of data assets, including the management of the data catalog for reference
• Draft architectural diagrams, interface specifications and other design documents