Senior Software Engineer – Python / PySpark (Mainframe Modernization)

Title: Senior Software Engineer - Python/PySpark

Location: San Francisco, CA
Duration: 24 months

Work Engagement: W2

Work Schedule: 3 days in office/2 days remote

Benefits on offer for this contract position: Health Insurance, Life insurance, 401K and Voluntary Benefits

Summary:

In this contingent resource assignment, you may: Consult on complex initiatives with broad impact and large-scale planning for Software Engineering. Review and analyze complex multi-faceted, larger scale or longer-term Software Engineering challenges that require in-depth evaluation of multiple factors including intangibles or unprecedented factors. Contribute to the resolution of complex and multi-faceted situations requiring solid understanding of the function, policies, procedures, and compliance requirements that meet deliverables. Strategically collaborate and consult with client personnel.

We are seeking a Senior Software Engineer with deep backend expertise to join a Scrum-based delivery team modernizing a mission-critical financial system. This role is heavily backend-focused, with strong emphasis on Python and PySpark, while also contributing to frontend development using React and Angular as needed.
This individual will serve as a technical SME, guiding a team that is actively upskilling from mainframe technologies into modern Python-based platforms. While this is not a people-lead role, the position requires strong mentorship, technical guidance, and exceptional communication skills. The engineer will work closely with Product Owners, Lines of Business, and Operations teams, and must be comfortable presenting technical concepts to both technical and non-technical audiences.

Key Responsibilities:

Design, develop, and modernize backend services to replace legacy mainframe functionality on a distributed platform
Build and maintain data ingestion, validation, and transformation pipelines using Python and PySpark
Develop and support APIs using FastAPI and/or Flask
Collaborate within a Scrum team, partnering closely with Product Owners, LOB stakeholders, and Operations
Act as a Python SME, mentoring engineers transitioning from mainframe technologies
Contribute to frontend development using React and Angular when required
Ensure data accuracy and integrity for financial feeds flowing into general ledger systems
Participate in technical design discussions, code reviews, and architecture decisions
Communicate effectively through documentation, presentations, and stakeholder discussions
Support CI/CD pipelines and cloud-native deployments using containerized platforms

Required Qualifications:

• Applicants must be authorized to work for ANY employer in the U.S. This position is not eligible for visa sponsorship.

• 7 years of relevant professional software engineering experience

• Strong expertise in Python, with production experience using PySpark

• Backend experience with FastAPI and/or Flask

• Frontend experience with React and/or Angular

• Experience with SQL Server and/or Exadata

• Hands-on experience with Kafka or other streaming platforms

• Experience working in Agile / Scrum delivery models

• Strong communication, presentation, and stakeholder engagement skills

• Ability to guide and mentor other engineers as a senior technical contributor

Preferred Qualifications:

• Agentic AI frameworks: LangGraph, LangChain, A2A

• Programming Languages: Java, Scala, SQL, HiveQL

• Big Data Technologies: Hadoop, Spark, HDFS, Hive, Cloudera, Hortonworks

• Cloud Platforms: AWS (Glue, Lambda, Redshift, S3, CloudWatch)

• ETL / ELT Tools: AWS Glue, PySpark, Databricks

• Data Integration & Migration: Hadoop ecosystems, Kafka, real-time streaming, data lakes

• Data Modeling: dimensional models, structured and unstructured data processing

• CI/CD & Automation: GitHub Actions, Azure DevOps, Jenkins, Git, Autosys, Airflow

• Containerization & Orchestration: Kubernetes