Lead

Lead

Primary Skills

Azure DevOps, Azure Functions, Event Hub, Cosmos DB, AKS

Specialization

Azure Data Engineering Advanced: Senior Data Engineer

Job requirements

The Role Senior Data Engineer to contribute towards the success of our Data and Analytics Technology initiatives. 
This is a hands-on technical role, and this person will be responsible for the development and build of scalable solutions as it relates to the architecture strategies, data standards, digital data management, data integration, tools, and technology. 
The right candidate will play a deep dive hands-on critical development role in the digital transformation and in shaping how we acquire, ingest, transform, and deliver data through the digital consumer channels. 
Key Responsibilities 
Architect, design, develop and engineering end-to-end data pipelines across multiple data sources and systems of record. 
Ensure data quality, integrity, security, and completeness throughout the data lifecycle. 
Develop, design data models, data structures and ETL jobs for data acquisition and transformation purposes. 
Develop deep understanding of the data sources, implement data standards, maintain data quality and master data management. 
Developing data services and API 
Work closely with the cloud service providers to ensure completeness and alignment with the service offerings. 
Manage and maintain cloud-based data and analytics platform. 
Deep understanding of the cloud offerings and engage in quick proof of concepts and proof of value in prototyping data and analytics solutions and derive viability. 
Understanding of Unified Data governance and Data Quality Management solutions for managing and governing enterprise data. 
Ability to interact with the business stakeholders to understand requirements and translating into technology solutions.

Technical Qualifications

Bachelor’s degree in computer science, computer engineering, or other related field of study with 10 years of relevant experience 
Experience in Azure Cloud platform service offerings 
Expertise in Azure SQL with strong knowledge on writing complex SQL queries, Stored procedures, Tuning and Query optimizations. 
Experience with Azure API Management Service (APIMS) and developing Azure Functions for exposing API / RESTful data services. 
Experience working on real-time data capture, processing, transformation and storing using technologies like Azure Event Hubs, Azure Service Bus, and Azure Stream Analytics 
Knowledge of creating pipelines and complex data transformations and manipulations using one of the languages .Net/C#, Python, Java, R, or Scala with Databricks/Spark 
Experience in NoSQL Databases and Big data technologies including Azure CosmosDB or other Document stores. 
Experience with Azure Purview for automating and managing Data catalog, classification, lineage, and metadata management across enterprise data. 
Experience with configuring rules, exceptions and approvals workflows for Data quality management using Profisee. 
Experience working with different data storage options including ADSLS Gen2 and Azure BLOB storage. 
Understanding of different data formats including Parquet, Avro, CSV, ORC etc. 
Understanding of MPP databases and maintain large amount of data processing is a plus. 
Experience in creating, scheduling, and monitoring Azure Data Factory (ADF) pipelines. 
Experience with Azure Synapse Analytics and Azure Analysis Services, Power BI and business Intelligence technologies is a big plus but not mandatory. 
Past working experience on a fast paced and agile environment 
Experience working with an on-shore / off-shore model. 
Build and meet project timelines and manage delivery commitments with proper communication to management. 
Collaborate and communicate with key business lines, technology partners, vendors, and architects. 
Equal Employment Opportunity Declaration Brillio is an equal opportunity employer to all, regardless of age, ancestry, colour, disability (mental and physical), exercising the right to family care and medical leave, gender, gender expression, gender identity, genetic information, marital status, medical condition, military or veteran status, national origin, political affiliation, race, religious creed, sex (includes pregnancy, childbirth, breastfeeding, and related medical conditions), and sexual orientation.