Top Skills & Years of Experience:
· 3 years of advanced hands-on experience designing AWS data lake solutions, integrating Redshift with other AWS services, such as DMS, Glue, Lambda, S3, Athena, and Airflow.
· Experience with PySpark and Glue ETL scripting including functions like renationalize, performing joins and transforming data frames with PySpark code.
Additional Useful Experience:
· Docker
· Airflow Server Administration
· Parquet file formats
· AWS Security
· Jupyter Notebooks
· API Best Practices, API Gateway, Route Structuring, and standard API authentication protocols including tokens.
· Git, Git flow best practices
· Release management and DevOps.
· Shell scripting
· AWS certifications related to data engineering or databases are a plus.
· Experience with DevOps technologies and processes.
· Experience with complex ETL scenarios, such as CDC and SCD logics, and integrating data from multiple source systems.
· Experience in converting Oracle scripts and Stored Procedures to Redshift equivalents.
· Experience working with large-scale, high-volume data environments.
· Exposure to higher education, finance, and/or human resources data is a plus.
Proficiency in SQL programming and Redshift stored procedures for efficient data manipulation and transformation.
Any gradudate