Title: Databricks/AWS architect
Location: Boston, MA (day 1 Onsite)
Day 1 consent from candidate is mandatory.
Job Description:
Key Responsibilities:
- Ability to design/develop based on business requirements by effectively coordinating activities between business analysts, developers, and testing team.
- Uses advanced technical expertise in analyzing, designing, estimating, and developing software applications to project schedule.
- Responsible for development work primarily related to integrating vendor application which is hosted SS environment.
- Independently undertakes critical development assignments across project areas.
- Participate in testing process – Perform in unit test case automation and execution in lower regions.
- Prepare Technical & functional design documents.
Required Qualifications:
- Must have minimum 10 number of years’ experience in software development.
- Engineering degree in Computer Science or equivalent field from top schools.
- Experience working in Data Integration projects.
- Hands on experience with AWS Cloud, Databricks, Unity Catalog, Python, PySpark, Scala, HIVE, Bigdata/Hadoop, and Shell scripts is required.
- Designing the Data Pipeline using Databricks clusters with AWS platform and services like AWS S3, RDS.
- Good working knowledge of Amazon Web Services (AWS) Cloud Platform which includes services like EC2, S3, ELB, Auto Scaling, EC2 Container Service (ECS)…etc.
- Hands-on experience in CICD & Jenkins is required.
- Solid understanding of Databricks fundamentals/architecture and have hands on experience in setting up Databricks cluster, working in Databricks modules (Data Engineering, ML and SQL warehouse).
- Experience in migrating data from on-prem Hadoop to Databricks/AWS
- Solid knowledge on Airflow.
- Excellent communication skills both verbal and written.
- Experience working in a geographically spread-out team.
- Outstanding coding, debugging and analytical skills: Core problem solving skills, ability to analyze available data and potential solutions, eliminate possible solutions and select an optimal solution.