What are the responsibilities and job description for the Sr. PySpark Engineer position at NuStar Technologies?
Must Have Technical/Functional Skills
- 10 years of experience in big data and distributed computing.
- Very Strong hands-on experience with PySpark, Apache Spark, and Python.
- Strong Hands on experience with SQL and NoSQL databases (DB2, PostgreSQL, Snowflake, etc.).
- Proficiency in data modeling and ETL workflows.
- Proficiency with workflow schedulers like Airflow
- Hands on experience with AWS cloud-based data platforms.
- Experience in DevOps, CI/CD pipelines, and containerization (Docker, Kubernetes) is a plus.
- Strong problem-solving skills and ability to lead a team
Salary : $100,000 - $120,000