Data Engineer

Must-have skills: 7+ y exp. building and maintaining ETL pipelines with large data sets using services such as AWS Glue, EMR, Kinesis or Kafka, CloudWatch; Python development experience with proficiency in Spark or PySpark and in using API; AWS Services such as AWS Lambda, Event Bridge, Step functions, SNS, SQS, S3 and MI models; Knowledge of machine learning, deep learning AMIs and natural language processing (NLP) techniques.
PREFERRED : Cloud Formation; in CI/CD pipeline and agile methodologies

Requirements:
• Availability to work 100% of the time at the Client’s site in Huston, TX (required);
• Hands-on experience working in data warehousing, data engineering and building machine learning models for solving complex problems using AWS services (7+ years);
• Experience building and maintaining ETL pipelines with large data sets using services such as AWS Glue, EMR, Kinesis or Kafka, CloudWatch;
• Experience with Python development with proficiency in Spark or PySpark and in using APIs;
• Experience in writing SQL queries and performance tuning in AWS Redshift and other industry leading RDMS such MS SQL Server, Postgres;
• Experience working with AWS Services such as AWS Lambda, Event Bridge, Step functions, SNS, SQS, S3 and MI models;
• Experience with machine learning, deep learning AMIs and natural language processing (NLP) techniques;
• Experience with Design and build machine learning models and algorithms to solve complex problems;
• Experience with how IAM Roles and Policies work

Preferred:
• Experience with infrastructure coding such as Cloud Formation;
• Experience in CI/CD pipeline and agile methodologies

Responsibilities include but are not limited to the following:
• Responsible for the development and integration of new or existing applications into the technical infrastructure and existing business processes;
• Provides technical or functional guidance to project or work teams as needed within a specific discipline;
• Collaborates on an on-going basis with the Business Systems Analyst;
• Analyzes, designs, develops, tests, debugs, implements, maintains and/or enhances existing or new systems that are reliable and efficient.