Senior Data Engineer (Databricks)

Senior Data Engineer (Databricks)

Fractal is a strategic AI partner to Fortune 500 companies with a vision to power every human decision in the enterprise. Fractal is building a world where individual choices, freedom, and diversity are the greatest assets. An ecosystem where human imagination is at the heart of every decision. Where no possibility is written off, only challenged to get better. We believe that a true Fractalite is the one who empowers imagination with intelligence. Fractal has been featured as a Great Place to Work by The Economic Times in partnership with the Great Place to Work® Institute and recognized as a ‘Cool Vendor’ and a ‘Vendor to Watch’ by Gartner.

Please visit Fractal | Intelligence for Imagination for more information about Fractal.

Note: This position is not eligible for Immigration Sponsorship at this time.

Role Overview

We are seeking a Senior Data Engineer for financial services who will be responsible for designing, building, and optimizing large‑scale data pipelines and cloud‑native data processing frameworks on AWS and Databricks. The role focuses on developing high‑performance ETL/ELT workflows, enabling reliable data movement, and ensuring data quality, security, and operational excellence across enterprise data platforms. This engineer collaborates with architects, modelers, analysts, and platform teams to deliver scalable, efficient, and well‑governed data solutions that support analytics, reporting, and machine learning workloads.

Requirements

Core Technical Expertise

• Proven experience designing and developing ETL/ELT pipelines and data migration workflows
• Strong proficiency in PySpark and distributed data processing frameworks
• Hands‑on experience with AWS data services (S3, Glue, Redshift, Lambda, EMR, Athena)
• Strong experience with Databricks (Delta Lake, Spark, workflows)
• Experience with Snowflake and Oracle for analytical and operational workloads
• Familiarity with monitoring and automation tools (e.g., CloudWatch, Grafana, Datadog)

Data Engineering & Architecture Skills

• Strong understanding of data architecture, data modeling concepts, and performance optimization
• Experience building and optimizing large‑scale data processing solutions
• Experience designing and managing data‑marts and analytical data structures
• Ability to implement data quality checks, validation frameworks, and governance controls
• Experience with workflow orchestration and automation tools (Airflow, Glue Workflows, Step Functions)

Operational & Platform Skills

• Experience implementing monitoring, logging, and observability for data pipelines
• Ability to optimize compute, storage, and query performance across cloud platforms
• Understanding of security, access control, encryption, and compliance requirements
• Experience working with CI/CD pipelines and version‑controlled data engineering workflows

Collaboration & Problem-Solving

• Strong analytical and troubleshooting skills
• Ability to work effectively in fast‑paced, cross‑functional environments
• Strong communication skills with the ability to collaborate with architects, analysts, and business teams

Roles & Responsibilities

Data Pipeline Development

• Design, develop, and maintain ETL/ELT pipelines for batch and streaming data
• Build and optimize PySpark‑based data processing solutions for large‑scale datasets
• Develop and manage data‑marts using Snowflake, Oracle, and AWS‑based data platforms

Cloud & Platform Engineering

• Implement and monitor data solutions using AWS services (S3, Glue, Lambda, EMR, Redshift, Athena) and Databricks
• Ensure pipelines are scalable, reliable, secure, and cost‑efficient
• Implement monitoring, alerting, and performance dashboards using Grafana or equivalent tools

Data Quality, Security & Governance

• Ensure data quality, consistency, and reliability across all data pipelines
• Implement security, access control, and compliance best practices
• Support metadata, lineage, and documentation standards

Collaboration & Delivery

• Work closely with data architects, modelers, and analysts to deliver end‑to‑end data solutions
• Participate in design reviews, code reviews, and optimization discussions
• Troubleshoot production issues and ensure high availability of data pipelines

Pay

The wage range for this role takes into account the wide range of factors that are considered in making compensation decisions including but not limited to skill sets; experience and training; licensure and certifications; and other business and organizational needs. The disclosed range estimate has not been adjusted for the applicable geographic differential associated with the location at which the position may be filled. At Fractal, it is not typical for an individual to be hired at or near the top of the range for their role and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range is: $110,000 - $150,000. In addition, for the current performance period, you may be eligible for a discretionary bonus.

Benefits

As a full-time employee of the company or as an hourly employee working more than 30 hours per week, you will be eligible to participate in the health, dental, vision, life insurance, and disability plans in accordance with the plan documents, which may be amended from time to time. You will be eligible for benefits on the first day of employment with the Company. In addition, you are eligible to participate in the Company 401(k) Plan after 30 days of employment, in accordance with the applicable plan terms. The Company provides for 11 paid holidays and 12 weeks of Parental Leave. We also follow a “free time” PTO policy, allowing you the flexibility to take time needed for either sick time or vacation.

Fractal provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.