What are the responsibilities and job description for the Pyspark Developer - NYC,NY or Dallas, TX (Onsite) position at Smart Caliber Technology?
Job Details
Role Name: Pyspark Developer
Location: NYC,NY or Dallas, TX (Onsite)
Duration: 6-12 Months
JOB DESCRIPTION: Must have skills : PySpark
Good to have skills : AWS Detailed JD :
• In-depth knowledge of Pyspark, DataFrames, Datasets, RDDs, Spark Streaming and Parquet partitioning concepts.
• Experience with Data bricks and AWS S2, AWS Athena, AWS Glue. Should have understanding for AWS Athena.
• This role requires strong skills in Python, PySpark, SQL, and experience with data warehousing and ETL/ELT processes. Senior developers also need to be proficient in cloud technologies and have experience with data governance and security.
• Data Python (must avoid Application Python)
• Strong experience with SQL.
• Knowledge of version control (Git), CI/CD tools, and Agile methodologies.
• Excellent communication.
• Should have the ability to train and lead the developers within the team, for any of the above skill gap.
• Should be ready to work from office all 5 days.
Best Regards,
Vishal
Truth Lies in Heart