Sr. Data Engineer

Job Description:

SUMMARY/OBJECTIVE

The Senior Data Engineer will execute on implementing business intelligence solutions across all divisions of the organization.
The individual in this role will guide the team in design and implementation of each solution and will implement improvements of technical standards, development processes, governance, and technologies.

ESSENTIAL DUTIES AND RESPONSIBILITIES:a

Serve as a technical lead in the development of business intelligence solutions. Coordinate with scrum masters / project managers on optimizing scheduling, work allocations, and story assignments.

Manage reporting environment including data sources, connections, security, subscriptions and metadata.

Assist in initiative towards data access control and integrity across the enterprise.

Create data models and schema for reliable and highly performant data marts and data warehouse.

Develops complex ETL (extract, transform and load) SQL scripts to gather data from various databases and systems to make data available for enterprise analytics and reporting.

Evaluate, test and implement new or upgraded software.

Troubleshoot reported problems pertaining to reports and the reporting database environment, working closely with other IT members, especially DBAs, Systems Analysts and Systems Engineers, identifying the cause and designing or developing an approach for resolution.

Assess impact of changes and updates to source production systems, planning and implementing resulting changes to reports.

POSITION DESCRIPTION

Identify areas for improvement within the EDS organization related to architecture, design, performance optimization, development process, workflows, governance, tools/technologies.

Conduct end user training on data solutions.

CORE COMPETENCIES

Forward Thinking: The ability to anticipate the implications and consequences of situations and take appropriate action to be prepared for possible contingencies.

Thinks Critically: The ability to approach one’s work analytically by identifying issues/questions, gathering and evaluating data, and creating evidence-based recommendations or solutions.

Embrace Innovation & Change The ability to respond positively to change and demonstrate flexibility, and openness to new ways of thinking and doing things.

Communicate Openly & Effectively: The ability to effectively communicate in multiple modes and in a way that fosters understanding and dialogue, as well as engages with others in a way that demonstrates emotional intelligence.

Performance Focused: The ability to take ownership for one’s own or one’s employees ’ performance, by setting clear goals and expectations, tracking progress against the goals, providing feedback, and addressing performance problems and issues promptly.

Apply Customer Lens: The ability to provide the best service to customers (internal and external) with the goal of exceeding their expectations while maintaining integrity.

Develop Self & Others: The ability to identify, learn, and apply new skills or habits as needed to perform successfully on the job.

Foster Collaboration: The ability to look beyond immediate teams, broaden peer networks, and finding ways to partner with internal and external stakeholders to achieve success.

EDUCATION, EXPERIENCE AND SKILLS

Bachelor Degree and/or relevant technical training with no preferred certification.

Over 8-10 years of experience in the field of Information Technology with proficiency in ETL design/ development and Data Warehouse Implementation/development

Experienced in Design, Development and Implementation of large - scale projects in Financial, Shipping and Retail industries using Data Warehousing ETL tools (Pentaho)

Knowledge about Software Development Lifecycle (SDLC), Agile, CI/CD deployment

Experience in Architecting and building Data Warehouse systems and Business Intelligence systems including ETL using Pentaho BI Suite (Pentaho Data Integration Designer / Kettle)

Hands on experience on the whole ETL (Extract Transformation & Load) process

Experience in creating ETL transformations and jobs using Pentaho Kettle Spoon designer and Pentaho Data Integration Designer and scheduling

Used various steps in Pentaho transformations including Row Normalizer, Row Demoralizer, Database Lookup, Database Join, Calculator, Add Sequence, Add Constants and various types of inputs and outputs for various data sources including Tables, Access, Text File, Excel and CSV file

 Integrating Kettle (ETL) with Hadoop and other various NoSQL data and loaded unstructured data into Hadoop File System (HDFS)

Experience in performing Data Masking/Protection using Pentaho Data Integration (Kettle)

Experience in writing shell scripting for various ETL needsExceptional data analytical abilities

Exceptional communication skills and collaborative problem-solving skills

Strong analytical and project management skills; attention to detail