AWS Engineer

Job Description

Required:

AWS, Kafka, Python, Terraform. Stream Processing application development experience using Kafka Streams and/or Flink APIs. Experience with Postgres, MySQL, DocumentDB.

Nice To Have

Flink, EMR, Hudi, AWS-Redshift, ECS/EKS, Glue Catalog – at least Solution Architect Associate, Developer, or Data Engineering certification preferred.

Roles & Responsibilities

Support or collaborate with application developers, database architects, data analysts and data scientists to ensure optimal data delivery architecture throughout ongoing projects/operations. 
Design, build, and manage analytics infrastructure that can be utilized by data analysts, data scientists, and non-technical data consumers, which enables functions of the big data platform for Analytics. 
Develop, construct, test, and maintain architectures, such as databases and large-scale processing systems that help analyse and process data in the way the Analytics organization requires. 
Develop highly scalable data management interfaces, as well as software components by employing programming languages and tools. 
Design, document, build, test and deploy data pipelines that assemble large complex datasets from various sources and integrate them into a unified view. 
Identify, design, and implement operational improvements: automating manual processes, data quality checks, error handling and recovery, re-designing infrastructure as needed. 
Create data models that will allow analytics and business teams to derive insights about customer behaviours. 
Build new data pipelines, identify existing data gaps and provide automated solutions to deliver analytical capabilities and enriched data to applications. 
Responsible for obtaining data from the System of Record and establishing batch or real-time data feed to provide analysis in an automated fashion. 
Develop techniques supporting trending and analytic decision-making processes 
Apply technologies for responsive front-end experience 
Ensure systems meet business requirements and industry practices 
Research opportunities for data acquisition and new uses for existing data 
Develop data set processes for data modelling, mining and production 
Integrate data management technologies and software engineering tools into existing structures 
Employ a variety of languages and tools (e.g. scripting languages)

What are the top 3 Must Haves for this role?

AWS 
Kafka 
Python

Additional Information

Provides technical direction, guides the team on key technical aspects and responsible for product tech delivery 
Lead the Design, Build, Test and Deployment of component 
Where applicable in collaboration with Lead Developers (Data Engineer, Software Engineer, Data Scientist, Technical Test Lead) 
Understand requirements / use case to outline technical scope and lead delivery of technical solution 
Confirm required developers and skillsets specific to product 
Provides leadership, direction, peer review and accountability to developers on the product (key responsibility) 
Works closely with the Product Owner to align on delivery goals and timing 
Assists Product Owner with prioritizing and managing team backlog 
Collaborates with Data and Solution architects on key technical decisions 
The architecture and design to deliver the requirements and functionality