Pyspark Tech Lead

Must Have: Python, Pyspark, NVIDIA Clara and SQL

Job Description

Relevant Experience (in Yrs) 10+

Technical/Functional Skills Python, Pyspark, NVIDIA Clara and SQL

Experience Required

Bachelors Degree with a minimum of 10+ years of relevant experience or equivalent.

8+ years of experience as Python, PySpark, SQL developer

Proficiency in writing Python/PySpark code from scratch to read data from different data sources and load into target databases (SQL / MongoDB) after applying transformation logics.

A solid understanding of SQL, rational databases, and normalization.

Extensive experience in developing, maintaining, and managing Python code.

Connecting to data sources, importing data and transforming data for Business Intelligence.

Experience on Python application refactoring using PySpark and NVIDIA Clara

Roles & Responsibilities

Coordinating with development teams to determine application requirements.

Work on client projects to deliver Python, PySpark based Data engineering Analytics solutions Build and operate very large data warehouses or data lakes ETL optimization, designing, coding, Build data pipelines

applications to stream and process datasets at low latencies Show efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data

Expertise in refactoring Python code to PySpark

Writing scalable code using Python programming language.

Testing and debugging applications.

Developing back-end components.

Integrating user-facing elements using server-side logic.

Assessing and prioritizing client feature requests.

Integrating data storage solutions.

Coordinating with front-end developers.

Reprogramming existing databases to improve functionality.

Developing digital tools to monitor online traffic