What are the responsibilities and job description for the Spark Developer - HYBRID - ONSITE Interview position at CCS IT?
Job Details
Key Responsibilities:
Migrate Cascading, Hadoop, and MapReduce workflows to Spark 3.
Optimize data pipelines using Spark 3 features like Adaptive Query Execution
(AQE) and Dynamic Partition Pruning.
Develop scalable solutions integrated with big data platforms and cloud
environments.
Align modernization initiatives with infrastructure teams and organizational goals.
Required Skills:
3 years of experience with Apache Spark (Spark 3.x required).
Hands-on experience transitioning from Cascading, Hadoop, or MapReduce to
Spark 3.
Proficiency in Scala, Python, or Java.
Strong understanding of Hadoop, Hive, and Kafka.
Expertise in profiling and optimizing Spark jobs.
Familiarity with AWS (EMR, Glue, S3) or Azure (Databricks, Data Lake).