Java Spark Engineer

Dear Applicant,

Please let me know if you are interested.

Position: Java Spark Engineer

Location: Berkeley heights, NJ (5 days onsite per week)

Hire Type: long term Contract

Job Description:

Responsibilities:

• Architect and build scalable, fault-tolerant data pipelines using Apache Spark (Java)

• Lead design of batch and streaming ETL/ELT systems handling large data volumes

• Deep-dive performance tuning: partitioning strategy, memory management, shuffle/skew optimization, job cost reduction

• Set coding standards and lead code/design reviews across the team

• Drive technical decisions on data architecture, storage formats, and pipeline orchestration

• Mentor mid-level and junior engineers; act as a technical escalation point

• Partner with product, analytics, and platform teams to translate requirements into scalable systems

• Own production reliability — on-call ownership, incident response, root-cause analysis for pipeline failures

• Evaluate and introduce new tools/frameworks where they improve the system

• Contribute to capacity planning and cost optimization for cluster infrastructure

Required Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field

• 7 years of professional Java development experience

• 5 years hands-on experience with Apache Spark in production environments

• Expert-level understanding of distributed systems: fault tolerance, data locality, shuffle mechanics, resource management

• Proven track record designing systems processing terabyte scale data

• Strong SQL skills and deep familiarity with columnar storage formats (Parquet, ORC, Avro, Delta Lake/Iceberg)

• Experience with cluster managers (YARN, Kubernetes) and cloud-managed Spark

• Proficiency with Kafka

• Strong grasp of CI/CD, containerization, and infrastructure-as-code practices

Preferred Qualifications

• Experience with Flink or other stream-processing frameworks

• Familiarity with data governance, lineage, and quality frameworks

• Experience with workflow orchestration at scale

• Background in system design for multi-tenant or multi-region data platforms

• Prior experience leading a team or acting as a technical lead

Soft Skills / Leadership

• Excellent communication — able to explain technical tradeoffs to non-technical stakeholders

• Strong mentorship and coaching ability

• Comfortable driving ambiguous, cross-team technical initiatives

Behavioral Skills:

1. Good Communication skills
2. 5 days Work from Office at Berkley Heights, NJ
3. Team Player
4. Ability to work in a changing environment
5. Strong problem solving and analytical skills
6. Ability to work independently or within a team
7. Manage day-to-day challenges and communicate developmental risks with the technical team