Azure Data Engineer

  Create and maintain optimal data pipeline architecture for data intensive applications.
•            Assemble large, complex data sets that meet functional / non-functional business requirements.
•            Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
•            Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Azure SQL, Cosmo DB, Databricks and other legacy databases.
•            Build analytics Dashboard/Visualizations utilizing the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
•            Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
•            Keep our data separated and secure across national boundaries through multiple data centers and Azure regions.
•            Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
•            Work with data and analytics experts to strive for greater functionality in our data systems.

Qualifications for Data Engineer:
•            Strong python programming skills, expert level on using Python to process Big Data;
•            Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
•            Extensive Experience on Databricks on Azure Cloud platform, deep understanding on Delta Lake, Lake House Architecture.
•            Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
•            Strong analytic skills related to working with Data Visualization Dashboard, Metrics and etc, experience on Tableau, Power BI or Looker tools;
•            Build processes supporting data transformation, data structures, metadata, dependency and workload management.
•            A successful history of manipulating, processing and extracting value from large disconnected datasets.
•            Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
•            Familiar with Deployment tool like Docker and building CI/CD pipelines.
•            Experience supporting and working with cross-functional teams in a dynamic environment.
•            8+ years’ experience in software development, Data engineering, and