·TheData Engineer will collaborate with product owners, developers, database architects,data analysts, visual developers and data scientists on data initiatives andwill ensure optimal data delivery and architecture is consistent throughoutongoing projects.

·Mustbe self-directed and comfortable supporting the data needs of the productroadmap. The right candidate will be excited by the prospect of optimizing andbuilding integrated and aggregated data objects to architect and support ournext generation of products and data initiatives.

·Createand maintain optimal data pipeline architecture,

·Assemblelarge, complex data sets that meet functional / non-functional businessrequirements.

·Identify,design, and implement internal process improvements: automating manualprocesses, optimizing data delivery, re-designing for greater scalability

·Comprehensivedocumentation and knowledge transfer to Production Support

·Workwith Production Support to analyze and fix Production issues

·Participatein an Agile / Scrum methodology to deliver high -quality software releasesevery 2 weeks through Sprint

·Refine,plan stories, and deliver timely

Requirements

·5 years’ experience in Big Data stack environments like AWS EMR, Clourdera,Hortonworks

·3 years of SPARK in batch mode

·3 years of experience in scripting using Python

·3 years of experience working on AWS Cloud environment

·In-depthknowledge of Hive and S3

·Strongunderstanding of Hadoop, MPP systems and data structures

·Strongunderstanding of solution and technical design

·Experiencebuilding cloud scalable high-performance data lake solutions

·Experiencewith relational SQL & tools like Snowflake

·Awareof Datawarehouse concepts

·Performancetuning with large datasets

·Experiencewith source control tools such as GitHub and related dev processes

·Experiencewith workflow scheduling tools like Airflow

·Strongproblem solving and analytical mindset