What are the responsibilities and job description for the Apache Spark Developer position at FDM Group?
About The Role
This position requires the successful candidate to work on a W2 directly with FDM. We cannot accept C2C, 1099 or employment sponsorship (e.g. H1-B) for this position.
FDM is a global business and technology consultancy delivering client and industry driven solutions through our 5 core specialist Practices; Software Engineering, Data & Analytics, IT Operations, Change & Transformation, and Risk, Regulation & Compliance.
FDM is seeking a located in Jersey City to support a project in the banking sector. Involvement in this project is anticipated to last initially 12 Months but may be extended.
This role will be hybrid with requirements to be in office 3 days per week.
About You
Senior Apache Spark Developer:
We are seeking a Senior Apache Spark Developer to help build and expand our next-generation data processing platform. This role will focus on designing and developing distributed data processing solutions using Apache Spark while leveraging an existing enterprise Kafka ecosystem.
The successful candidate will play a critical role in scaling our data processing capabilities, enabling cloud adoption, and supporting strategic migration initiatives involving Databricks and AWS. This position offers the opportunity to work on high-volume, mission-critical data platforms processing large-scale real-time and batch workloads.
Key Responsibilities
- Design, develop, and optimize Spark-based applications for large-scale data processing.
- Build streaming and batch data solutions leveraging Apache Kafka and Apache Spark.
- Develop reusable frameworks and components supporting enterprise data engineering initiatives.
- Support migration efforts from on-premises data platforms to Databricks and AWS.
- Collaborate with architects and platform engineers to implement cloud-native data solutions.
- Optimize Spark jobs for performance, scalability, reliability, and operational efficiency.
- Participate in system design, code reviews, testing, and production support activities.
- Implement CI/CD, monitoring, and observability practices for distributed data applications.
- Troubleshoot and resolve complex issues across Spark, Kafka, Databricks, and cloud environments and with other enterprise messaging systems.
Required Qualifications
- 5 years of hands-on development experience with Apache Spark.
- Strong expertise in:
- Spark SQL
- Structured Streaming
- DataFrames/Datasets
- Performance tuning and optimization
- Experience integrating Spark with Apache Kafka.
- Strong development skills in:
- Java
- Python
- Experience designing and supporting large-scale distributed systems.
- Solid understanding of software engineering principles, object-oriented design, and testing practices.
- Experience working with relational and distributed data platforms.
- Bachelor’s Degree
Preferred Qualifications
- Databricks experience (SQL).
- AWS experience, including services such as:
- S3
- EMR
- Glue
- Lambda
- ECS/EKS
- Experience supporting large-scale cloud migration initiatives.
- Experience with lakehouse architectures and modern data platform patterns.
- Experience handling multi-terabyte or petabyte-scale datasets.
- Financial services or regulated industry experience.
About Us
About FDM
FDM powers the people behind tech and innovation. We spot trends, find top talent, and help businesses stay ahead.
With 35 years of experience, we coach, mentor, and launch fresh thinkers from diverse backgrounds into world-class careers. Partnering with top global companies, we deliver the right talent at the right time—while guiding our people toward exponential growth.
🌍 Global impact – 18 centers across North America, APAC, the UK, and Europe
🚀 25,000 careers launched – and counting
🤝 300 trusted client partners
Committed to Equal Opportunities
Tech careers should be for everyone. With 75 nationalities represented, FDM thrives on diversity, fuels innovation through unique perspectives, and celebrates success together. As an Equal Opportunity Employer and FTSE4Good-listed company, we ensure every qualified applicant gets a fair shot—no barriers, just opportunities.
Additional Considerations
FDM Group, Inc. is registered to operate and hire employees in select states within the US. We will consider employment applications exclusively from candidates who are either residing in one of the following states or willing to relocate to them: Arizona, California, Colorado, Connecticut, Delaware, Florida, Georgia, Illinois, Indiana, Massachusetts, Maryland, Minnesota, North Carolina, New Jersey, New York, Pennsylvania, Tennessee, Texas, Utah, and Virginia.