Lead Big data Engineer

Role: Lead Big data Engineer

Location: Jersey City, NJ Onsite 

Job type: C2C

Must-Have Capital market Experience

Primary Tech skills – AZURE, PySpark, SQL, ADF

Secondary Tech skills –  SAS, Synapse, Databricks

Roles: 

•Technical coordination with the Product Group Technical Lead, Architecture team, Data engineering team, BSA, and QE Team for the assigned projects. 

Ability to drive more than one projects in parallel.

•Experience of DWH, Data Integration, Cloud, Architecture, Design, Data Modelling

•Experience in PySpark,  SQL, Cloud data warehouse like Azure Synapse, databricks

•Experience working with structured and unstructured data

•Extensive knowledge of ETL and Data Warehousing concepts, strategies, methodologies

•Experience in ETL/Pipeline Development using tools such as Azure Databricks and Azure Data Factory with development expertise on batch and real-time data integration

•Experience in data ingestion, preparation, integration, and operationalization techniques in optimally addressing the data requirements

•Experience in relational data processing technology like MS SQL, Delta Lake, Spark SQL, SQL Server

•Experience with Orchestration tools, Azure DevOps, and GitHub

•Experience to own end-to-end development, including coding, testing, debugging, and deployment

•Must be team-oriented with strong communication, collaboration, prioritization, and adaptability skill