Platform Engineer

Job Description:

Responsibilities

Architecture and Design:

• Provide input into the strategic technology direction for our data platform.
• Architect scalable and efficient data solutions using Azure Databricks.

Data Pipelines and Integration:

• Design, implement, and manage data pipelines.
• Ensure seamless data integration across various systems.
• Optimize data processing workflows.

Infrastructure Automation:

• Develop Terraform Infrastructure as Code (IaC) to automate Databricks environment deployments.
• Create a library of scripts for automating MLOps environments.

Performance Monitoring and Tuning:

• Monitor and analyze database performance.
• Fine-tune and optimize the data platform.

Collaboration and Communication:

• Work closely with interdisciplinary teams, including engineers, data scientists, and business stakeholders.
• Communicate effectively to ensure alignment with business goals.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related field.
• Proven experience as a Platform Engineer or similar role.
• Strong expertise in Databricks and Microsoft Azure.
• Experience with Infrastructure as Code (IaC) and automation tools, preferably Terraform.
• Knowledge of data catalog tools (e.g., Microsoft Purview).
• Excellent problem-solving skills and ability to work in a collaborative environment

Skill: "Platform Engineer" AND Azure AND Databricks AND IAC