Databricks Platform Architect

Databricks Platform Architect

Fractal is a strategic AI partner to Fortune 500 companies with a vision to power every human decision in the enterprise. Fractal is building a world where individual choices, freedom, and diversity are the greatest assets. An ecosystem where human imagination is at the heart of every decision. Where no possibility is written off, only challenged to get better. We believe that a true Fractalite is the one who empowers imagination with intelligence. Fractal has been featured as a Great Place to Work by The Economic Times in partnership with the Great Place to Work® Institute and recognized as a ‘Cool Vendor’ and a ‘Vendor to Watch’ by Gartner.

Please visit Fractal | Intelligence for Imagination for more information about Fractal.

Note: This position is not eligible for Immigration Sponsorship at this time.

Role Overview

We are seeking a Platform Architect for Financial Services who would be responsible for designing, governing, and scaling enterprise‑grade data models and foundation data architecture on AWS and Databricks. The role ensures that data is modeled, governed, and optimized to support analytical, operational, and AI/ML workloads. This architect collaborates with engineering, governance, and business teams to establish modeling best practices, ensure platform alignment, and enable scalable, high‑quality data platforms.

Requirements

Core Technical Expertise

• Strong experience in conceptual, logical, and physical data modeling
• Hands‑on experience with AWS data services (S3, Glue, Redshift, DynamoDB, Athena, EMR) and Databricks
• Proficiency in SQL, Python, Spark, and modern data integration frameworks
• Experience with dbt Core/Cloud and Data Vault 2.0 (including automate_dv)

Architecture & Data Engineering Skills

• Experience with data integration patterns and model ETL/ELT pipelines
• Experience designing data provisioning layers for downstream consumption
• Familiarity with modeling for AI/ML, including feature structuring and model‑ready datasets

Platform & Cloud Skills

• Understanding of Databricks Lakehouse modeling patterns (Delta Lake, Unity Catalog)
• Experience optimizing models for performance, cost, and scalability
• Experience implementing observability, monitoring, and data quality validation

Collaboration & Leadership

• Ability to work across engineering, governance, and business teams
• Strong communication skills with ability to explain modeling decisions clearly
• Experience working in hybrid onsite/offshore delivery models
• Financial services experience preferred

Roles & Responsibilities

• Design and architect enterprise-grade data platforms using AWS services and Databricks Lakehouse Platform.
• Define platform architecture for data ingestion, processing, storage, governance, observability, security, and cost optimization.
• Build scalable and secure data lakehouse solutions using Databricks, Delta Lake, Unity Catalog, Spark, and AWS native services.
• Design and implement cloud infrastructure using AWS services such as S3, IAM, VPC, Glue, Lambda, EMR, Redshift, Athena, KMS, CloudWatch, and Step Functions.
• Lead platform setup, workspace configuration, cluster policies, access controls, networking, and environment segregation across dev, test, and production.
• Establish best practices for Databricks workspace architecture, Unity Catalog governance, job orchestration, CI/CD, and data security.
• Collaborate with data engineering, analytics, AI/ML, security, and infrastructure teams to deliver robust platform capabilities.
• Define reusable platform patterns, reference architectures, automation templates, and deployment standards.
• Drive implementation of Infrastructure as Code using tools such as Terraform, CloudFormation, or similar.
• Implement CI/CD pipelines for Databricks notebooks, jobs, workflows, libraries, and infrastructure deployments.
• Ensure platform compliance with enterprise security, data privacy, governance, audit, and regulatory requirements.
• Optimize Databricks clusters, Spark workloads, storage design, and AWS resource usage for performance and cost efficiency.
• Support migration and modernization of legacy data platforms to AWS and Databricks.
• Provide technical leadership, architecture reviews, design guidance, and mentoring to engineering teams.
• Work with stakeholders to translate business and technical requirements into scalable platform solutions.
• Evaluate new AWS and Databricks features and recommend adoption where relevant.
• Define monitoring, logging, alerting, incident response, and operational support processes for the platform.
• Create technical documentation, architecture diagrams, runbooks, standards, and governance guidelines.

Platform Architecture Support

• Partner with platform architects to align data models with cloud‑native platform design
• Support design of data ingestion, transformation, and provisioning patterns
• Ensure models align with security, compliance, and access‑control requirements
• Contribute to reference architectures and reusable modeling patterns

Engineering Collaboration

• Work closely with data engineers to ensure accurate implementation of models
• Provide modeling guidance during solution design reviews
• Support automation of modeling and metadata processes where possible

Quality, Optimization & Governance

• Ensure data models are optimized for cost, performance, and maintainability
• Contribute to governance frameworks for data standards and lifecycle management
• Support monitoring, observability, and quality validation for modeled datasets

Pay

The wage range for this role takes into account the wide range of factors that are considered in making compensation decisions including but not limited to skill sets; experience and training; licensure and certifications; and other business and organizational needs. The disclosed range estimate has not been adjusted for the applicable geographic differential associated with the location at which the position may be filled. At Fractal, it is not typical for an individual to be hired at or near the top of the range for their role and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range is: $125,000 - $175,000. In addition, for the current performance period, you may be eligible for a discretionary bonus.

Benefits

As a full-time employee of the company or as an hourly employee working more than 30 hours per week, you will be eligible to participate in the health, dental, vision, life insurance, and disability plans in accordance with the plan documents, which may be amended from time to time. You will be eligible for benefits on the first day of employment with the Company.In addition, you are eligible to participate in the Company 401(k) Plan after 30 days of employment, in accordance with the applicable plan terms. The Company provides for 11 paid holidays and 12 weeks of Parental Leave. We also follow a “free time” PTO policy, allowing you the flexibility to take time needed for either sick time or vacation.

Fractal provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.