Description

Required Skillset and Roles & Responsibilities:

  • Bachelor's or Master's degree in Computer Science, Engineering, or related field
  • 8+ years of experience in big data engineering, data warehousing, and data integration
  • 3+ years of experience in cloud-based big data platform development using Azure cloud services
  • Experience in developing CI/CD pipelines for Azure SQL, Azure Databricks, and Azure Data Factory
  • Experience in developing and managing big data processing pipelines using Apache Spark, Hadoop, and other big data technologies
  • Experience in developing and managing machine learning pipelines for building and deploying machine learning models on big data
  • Experience in developing and managing data visualization and reporting mechanisms
  • Design and develop big data platform architecture based on Azure cloud services such as Azure HDInsight, Azure Databricks, Azure Data Lake Store, and others
  • Design and implement data ingestion pipelines for ingesting data from various sources such as databases, APIs, and other sources
  • Develop and manage big data processing pipelines for performing data transformations, aggregations, and analytics using Apache Spark and other big data technologies
  • Design and implement data storage mechanisms based on Azure cloud services such as Azure Data Lake Store, Azure Blob Storage, and others
  • Work closely with other members of the team such as data scientists, data analysts, and software engineers to understand their requirements and integrate their solutions into the big data platform
  • Develop and implement test strategies, plans, and cases for Azure SQL, Azure Databricks, and Azure Data Factory
  •  

 

Technical Skills:

  • Expertise in Azure cloud services such as Azure HDInsight, Azure Databricks, Azure Data Lake Store, and others
  • Expertise in big data processing technologies such as Apache Spark, Hadoop, and others
  • Proficiency in programming languages such as Python, Java, Scala, and others
  • Experience in developing and managing CI/CD pipelines using tools such as Azure DevOps, Jenkins, and others
  • Experience in developing and managing data integration pipelines using tools such as Azure Data Factory, Apache Nifi, and others
  • Experience in developing and managing data storage mechanisms based on Azure cloud services such as Azure Data Lake Store, Azure Blob Storage, and others

Key Skills
Education

Bachelor’s or Master’s degree

Back To Jobs