What are the responsibilities and job description for the Big Data Developer position at SPADTEK SOLUTIONS?
Title: CPS Data CoE - Big Data Engineer
Location: Alpharetta, GA 30005
Experience: 8 – 10 years
Duration: 6months
Work Authorization: We are currently considering only H1B and H4‑EAD candidates for this role
Responsibilities:
Design & develop new automation framework for ETL processing
Support existing framework and become technical point of contact for all related teams
Enhance existing ETL automation framework as per user requirements
Performance tuning of spark, snowflake ETL jobs
New technology POC and suitability analysis for Cloud migration
Process optimization with the help of automation and new utility development
Work in collaboration for any issues and new features
Support any batch issue
Support application team teams with any queries
Required Skills:
7 years of Data engineering experience
Must be strong in UNIX Shell, Python scripting knowledge
Must be strong in Spark
Must have strong knowledge of SQL
Hands-on knowledge on how HDFS/Hive/Impala/Spark works
Strong in logical reasoning capabilities
Should have working knowledge of GitHub, DevOps, CICD/ Enterprise code management tools
Strong collaboration and communication skills
Must possess strong team-player skills and should have excellent written and verbal communication skills
Ability to create and maintain a positive environment of shared success
Ability to execute and prioritize a task and resolve issues without aid from direct manager or project sponsor
Good to have working experience on snowflake & any data integration tool i.e. informatica cloud
Primary Skills:
Python
Big Data
Apache Spark
Desired Skills:
Snowflake/Azure/AWS any cloud
IDMC/any ETL tool
Salary : $60 - $65