Google Cloud Data Engineer / Cloud Engineer

Job Details

Technical/Functional Skills:

Git and Jenkins for orchestrating automated workflows.

Google Cloud SDK

Infrastructure-as-code tools such as Ansible, Helm and Terraform

Terraform state manipulation

Dashboards (BigQuery, Cloud Pub/Sub, Cloud Storage, Dataflow, Disks, Load Balancers, VM Instances

GCP Build/release systems, CI/CD systems, Jenkins

Containerization and cluster management technologies such as Docker and Kubernetes

Templating engines such as Jinja, Go templates

SQL query writing

Data streaming tools such as Oracle GoldenGate and Apache Kafka

COTS reporting tools such as Cognos & Tableau

Automated testing and automated deployment, source version control

Google Cloud technology: Cloud Monitoring, Cloud Logging, Cloud Load Balancing, Cloud Persistent Disk

Roles & Responsibilities

Design pipelines and architectures for data processing

Design and secure re-usable, flexible, built-as-code data ingress & egress patterns

Build highly scalable, fault tolerant data ingress & egress patterns in support of data engineering, machine learning, and analytics projects

Build repeatable batch data pipelines in cloud (Extra-Load, Extract-Load-Transform or Extract-Transform-Load)

Work with business and data team to efficiently use Google Cloud platform to analyze data, build data models on Big query, big table

Develop data architectures and data migration to cloud strategy

Work with business SME to perform source system data analysis, data mappings and build data inventory

Integrate massive datasets from multiple data sources for data modelling

Implement methods for DevOps automation of all parts of the build data pipelines to deploy from development to production