Job Description
Health Care Company, Inc. (Health Care Company) is looking for a Senior Data Engineer to work 100% remote (will work 9am-5pm either AZ time zone or EST time zone) to design, develop, test, and maintain pipelines ingesting critical real-time claims data sets into OnePaaS (Cloud)
Accountable for data engineering lifecycle including research, proof of concepts, design, development, test, deployment, and maintenance
Design, develop, implement and run cross-domain, modular, optimized, flexible, scalable, secure, reliable and quality data solutions that transform data for meaningful analyses and analytics while ensuring operability
Design, develop, implement and run data solutions and implement data pipelines with petabyte scale of data.
Build processes and diagnostics tools to troubleshoot, maintain and optimize solutions and respond to customer and production issues.
Embrace continuous learning of engineering practices to ensure industry best practices and technology adoption, including DevOps, Cloud and Agile thinking
Maintain high quality documentation of data definitions, transformations, and processes to ensure data governance and security.
Identifies solutions to non-standard requests and problems.
Solves moderately complex problems and/or conducts moderately complex analyses.
Ingest data from Hadoop sources to cloud.
Design and develop ETL workflows for migration of data from on-premise to Azure cloud
Develop CI/CD workflows on Azure for continuous deployment for projects
Work on Databricks, Azure data factory to migrate SSIS workflows into cloud.
Requirements
5 years of experience on Azure cloud
Python or Scala experience
Azure Databricks
Azure Data Factory (ADF)
UI or API build experience (React or Angular)
Kafka for real-time integration
Bachelor's Degree
Desired Skills and Experience
AZURE DATABRICKS, AZURE DATA FACTORY, ADF, AZURE CLOUD, REACT, ANGULAR, KAFKA, PYTHON, SCALA
ANY GRADUATE