SAP DATA ARCHITECT

Understanding of Basics of SAP, Apache NiFi, AWS Cloud tools such as S3, Athena, Redshift, Cloud9, Data Pipeline, Kafka, Glue, Scala/Pyspark, EMR, JIRA

Working with Business team on daily updates

· Working with Technical PM on daily basis

· Coordinating with offshore AWS team

· understanding basics of SAP and/or ERP systems or BW/HANA analytics

· Design data ingestion and data processing pipelines using technologies such as Apache NiFi, Amazon MSK, Amazon EMR, PySpark, Scala, Athena, Glue and Data Pipelines to bring the data from Hana into Datalake that resides on Amazon S3.

· Design data ingestion and data processing pipelines to move the data from S3 into data warehouse built using Amazon Redshift.

· Develop design of data visualization layer that will be built using Amazon Redshift for Amazon Quicksight.

· Build data ingestion and data processing pipelines using the above-mentioned technologies to bring the data from Hana into Amazon S3.

· Working Knowledge of Python, JSON, Scala, Pyspark

· Knowledge to use to python programs to integrate with Spark and utilize AWS Cloud tools such as Data Pipeline, EMR etc.

· Knowledge of JSON programs to utilize EMR cluster to use the metadata to write pipelines

· Working knowledge on converting orc to parquet

· Build data ingestion and data processing pipelines to move the data from S3 into Amazon Redshift.

· Build datasets in Amazon Quicksight utilizing the Redshift Data. Create Analysis and publish dashboards in Quicksight using the datasets.