Data Engineer

Job Summary (Primary Purpose)
We are seeking a Data Engineer to join a team developing an enterprise data lake that supports data science, analytics, and business intelligence activities across the company. We are looking for someone to help build cloud data ingestion and processing pipelines using hybrid cloud technology stacks The ideal candidate will have extensive experience with working on data platforms on the AWS Cloud; experience with internet-of-things (IoT) and healthcare data is a bonus but not a requirement. The candidate will join a global team of 5-10 high-performing engineers building a data platform to support an expanding set of internal users and a growing volume and variety of data.

Key Responsibilities and Duties

• Set up and manage automated data pipelines to capture IoT, Healthcare, system operations and third-party data.
• Develop cloud-based data processing pipelines using Spark, SQL, Python, AWS Lambda functions, REST APIs Create data models to represent data at different layers of classfications used for analytics and business intelligence.
• Collaborate with other data engineers and application developers to prioritize and develop data solutions that serve the needs of the company.

Experience and Skills

• BS in computer science, mathematics, engineering, information systems, or related field with 5+ year of work experience as a data engineer.
• Solid foundations in software engineering and cloud infrastructure.
• Experience with Data Modeling and Modern Data Architecture.
• Experience building data integrations using different data source such as cloud storage buckets, REST API, JDBC database, Kafka Streams etc.
• Experience deploying cloud infrastructure and data pipelines using Terraform.
• Extensive experience with SQL, Python, Apache Spark/PySpark, ApacheSpark,Architecture and Performance tuning.
• Experience with cloud-based data platforms and technologies such as Databricks, Snowflake.
• Experience of using foundational AWS Services such as S3, IAM, AWS Lambda functions, Secrets Manager, Event Bridge.
• Experience of building stream processing pipelines with Kafka and Structured Spark.
• Streaming Familiarity with quality engineering practices including software testing and validation.
• Experience with verification testing of data accuracy and quality.
• High attention to detail Strong written and oral communication skills
• Ability to work under pressure in a startup environment.
• Experience working with Onsite / Offshore model.