What are the responsibilities and job description for the Big Data PySpark Developer position at Data Wave Technologies Inc?
Job Title: Big Data PySpark Developer
Location: New York - NY
Duration: Long term
Tax Term: W2
Job description
Experience Required
- 10 years of experience in Big Data development
- Strong Python and PySpark expertise
- Experience with Hadoop ecosystem
- Spark SQL, Hive, HiveQL
- Kafka and data pipeline development
- Strong SQL skills
- Unix/Linux environment
- Software Development Life Cycle (SDLC)
- Design, coding, unit testing, debugging, and deployment
- Ability to work independently and within Agile teams
Responsibilities
- Design, build, and deploy Big Data solutions
- Develop scalable data processing applications using PySpark
- Create and optimize Spark/Hive jobs
- Work with large-scale datasets
- Follow architecture, testing, and coding standards
- Troubleshoot production issues
- Collaborate with business and technical stakeholders. Deliver high-quality code with minimal defects