Sr. Data Engineer/ETL Engineer

The Senior Data Engineer will execute on implementing business intelligence solutions across all divisions of the organization. The individual in this role will guide the team in design and implementation of each solution and will implement improvements of technical standards, development processes, governance, and technologies.

 ESSENTIAL DUTIES AND RESPONSIBILITIES:

1. Serve as a technical lead in the development of business intelligence solutions.
2. Coordinate with scrum masters / project managers on optimizing scheduling, work allocations, and story assignments.
3. Manage reporting environment including data sources, connections, security, subscriptions and metadata.
4. Assist in initiative towards data access control and integrity across the enterprise.
5. Create data models and schema for reliable and highly performant data marts and data warehouse.
6. Develops complex ETL (extract, transform and load) SQL scripts to gather data from various databases and systems to make data available for enterprise analytics and reporting.
7. Evaluate, test and implement new or upgraded software.
8. Troubleshoot reported problems pertaining to reports and the reporting database environment, working closely with other IT members, especially DBAs, Systems Analysts and Systems Engineers, identifying the cause and designing or developing an approach for resolution.
9. Assess impact of changes and updates to source production systems, planning and implementing resulting changes to reports.

 POSITION DESCRIPTION:

1. Identify areas for improvement within the EDS organization related to architecture, design,
2. performance optimization, development process, workflows, governance, tools/technologies.
3. Conduct end user training on data solutions. 

EDUCATION, EXPERIENCE AND SKILLS:

1. Bachelor Degree and/or relevant technical training with no preferred certification.
2. Over 8-10 years of experience in the field of Information Technology with proficiency in ETL design/ development and Data Warehouse Implementation/development
3. Experienced in Design, Development and Implementation of large - scale projects in Financial,
4. Shipping and Retail industries using Data Warehousing ETL tools (Pentaho)
5. Knowledge about Software Development Lifecycle (SDLC), Agile, CI/CD deployment
6. Experience in Architecting and building Data Warehouse systems and Business Intelligence systems including ETL using Pentaho BI Suite (Pentaho Data Integration Designer / Kettle)
7. Hands on experience on the whole ETL (Extract Transformation & Load) process
8. Experience in creating ETL transformations and jobs using Pentaho Kettle Spoon designer and Pentaho Data Integration Designer and scheduling
9. Used various steps in Pentaho transformations including Row Normalizer, Row Demoralizer,
10. Database Lookup, Database Join, Calculator, Add Sequence, Add Constants and various types of inputs and outputs for various data sources including Tables, Access, Text File, Excel and CSV file
11. Integrating Kettle (ETL) with Hadoop and other various NoSQL data and loaded unstructured data into Hadoop File System (HDFS)
12. Experience in performing Data Masking/Protection using Pentaho Data Integration (Kettle)
13. Experience in writing shell scripting for various ETL needs
14. Exceptional data analytical abilities
15. Strong development experience in writing PL/SQL Stored Procedures, Functions, packages and debugging the code
16. Strong development experience working with Columnar DBs preferably HP Vertica
17. Strong expertise in Developing and Debugging MSSQL or PostgreSQL SQL scripts, functions, stored procedures, views, complex SQL queries, cursors, and triggers
18. Expertise in Performance tuning, Query optimization and exception handling