Description

Job Description:

As a Hadoop Developer, the ideal candidate for this developer role should be able to:

  • Build high performing data models on big-data architecture as data services.
  • Build a high performing and scalable data pipeline platform using Hadoop, Apache Spark.
  • Partner with Enterprise data teams such as Data Management & Insights and Enterprise Data Environment (Data Lake) and identify the best place to source the data
  • Work with business analysts, development teams and project managers for requirements and business rules.
  • Collaborate with the source system and approved provisioning point (APP) teams, Architects, Data Analysts and Modelers to build scalable and performant data solutions.
  • Effectively work in a hybrid environment where legacy ETL and Data Warehouse applications and new big-data applications co-exist
  • Work with Infrastructure Engineers and System Administrators as appropriate in designing the big-data infrastructure.
  • Work with DBAs in Enterprise Database Management group to troubleshoot problems and optimize performance
  • Support ongoing data management efforts for Development, QA and Production environments
  • Utilizes a thorough understanding of available technology, tools, and existing designs.
  • Leverage knowledge of industry trends to build best in class technology to provide a competitive advantage.
  • Provide QA support & do Testing as needed.

 

Required Qualifications

  • ETL (Extract, Transform, Load) Programming experience
  • Experience in Hadoop ecosystem tools for real-time batch data ingestion, processing, and provisioning such as Apache Flume, Apache Sqoop, Apache Spark
  • Java/Scala/Python experience
  • Agile experience
  • Good SQL knowledge
  • Design and development experience with columnar databases using Parquet or ORC file formats on Hadoop
  • Apache Spark design and development experience using Scala, Java, Python or Data Frames with Resilient Distributed Datasets (RDDs)

 

Desired Qualifications

  • Excellent verbal, written, and interpersonal communication skills
  • Ability to work effectively in a virtual environment where key team members and partners are in various time zones and locations
  • Knowledge and understanding of project management methodologies: used in waterfall or Agile development projects
  • Knowledge and understanding of DevOps principles
  • Reporting experience, analytics experience or a combination of both
  • A BS/BA degree or higher in information technology

Education

Any Graduate