Sr. Big Data Engineer

●  Build analytical solutions to enable Data Scientist by manipulating large data sets and integrating diverse data sources.
●  Perform ad-hoc analysis and develop reproducible analytical approaches to meet business requirements.
●  Perform exploratory and targeted data analyses using descriptive statistics and other methods.
●  Use complex algorithms to develop systems & applications that deliver business functions or architectural components.
●  Work closely with the data scientists, and database and systems administrators to create data solutions.
●  Improve existing data pipelines by simplifying and increasing performance.
●  Design, build, and deploy new data pipelines within Big Data Eco-Systems.
●  Documents new/existing pipelines, Data Sets and Data Sets.
●  Abides by department development standards and SOP's.
●  Experience of designing, building, and deploying production-level data pipelines using tools from Hadoop stack (HDFS, Hive, Spark, HBase, Kafka, NiFi, Oozie, Splunk etc).