Pyspark Developer

UST is looking for a Strong developer who has :

Strong Hands-on in AWS cloud in S3/ATHENA/GLUE/EMR/VPC

Creating and Managing Data Pipelines and Streamsets in KAFKA

5+ yrs of Proven experience as a PySpark Developer or a related role

Strong programming skills in Scala, Spark or Python

Experience in working with AWS cloud and AWS DocuDB

Familiarity with big data processing tools and techniques

Experience with streaming data platforms

Excellent analytical and problem-solving skills

Strong knowledge in Kafka topics and K-SQL

Extensive Unix/FTP/ File Handling

Strong Hands-on in No SQL databases like Mongo DB

Experience with Agile Methodology/JIRA/Confluence

Hands-on in Handling all Processors

Responsibilities

Extensive knowledge in understanding Data and creating Data Pipelines

Interact with Business to understand the requirement on a daily evolving need and adapt to the needs

Handle streaming data platforms

Able to handle all the database storage and retrieval mechanisms for the Mongo DB on and extensive experience in handling

Able to understand the business requirements, tech stack involved and guide the team towards Integral solution approach

Provide extensive development support to the Team in achieving the Project Milestones

Dedicated and committed approach with excellent communication skills

Overall 8+ years of IT Development experience

5+ years of expereince as a PySpark Developer or a related role

Good experience in AWS, AWS Glue, Hudi & Kafka

Experience in working with AWS cloud, PostGreSQL and AWS DocuDB

Familiarity with big data processing tools and techniques

Experience with streaming data platforms

Excellent exposure with both relational and non-relation Databases

Experience in Nifi will be a plus

Excellent analytical and problem-solving skills