Sr Solution Architect - Azure DataBricks

Hello,

Hope you are doing good,

Position: Senior Solution Architect Azure Databricks

Location: Princeton, NJ (Remote)

Duration: Contract opportunity

Job Description:

We are seeking a highly experienced Techno-Functional Azure Databricks Architect with deep expertise in the Healthcare Payer domain and a proven track record of leading large-scale Enterprise Data Warehouse (EDW) to Azure Databricks migration programs. The ideal candidate will possess strong architecture, data engineering, stakeholder management, and delivery leadership skills, capable of driving digital transformation initiatives from strategy through execution.

This role requires a blend of technical architecture expertise, domain knowledge in healthcare payer operations, and program leadership capabilities including client engagement, work allocation, team mentoring, and governance management.

Key Responsibilities

Architecture & Solution Design

• Define end-to-end cloud data architecture and migration strategy for enterprise data modernization initiatives.
• Lead migration of legacy EDW platforms (Teradata, Netezza, Oracle, SQL Server, Hadoop, etc.) to Azure Databricks Lakehouse architecture.
• Design scalable, secure, and cost-optimized data platforms leveraging Azure services.
• Establish architectural standards, reusable frameworks, and best practices for data ingestion, transformation, and data consumption.

Healthcare Payer Domain Leadership

• Collaborate with business stakeholders to understand payer processes and data requirements.
• Translate healthcare business requirements into technical solutions supporting:
  • Claims Processing
  • Membership & Enrollment
  • Provider Management
  • Benefits Administration
  • Care Management
  • Utilization Management
  • Risk Adjustment
  • HEDIS/Quality Reporting
  • Regulatory Reporting

Data Migration & Modernization

• Lead legacy EDW assessment and migration planning activities.
• Define migration patterns, data validation frameworks, reconciliation strategies, and cutover plans.
• Oversee data profiling, data quality assessments, and remediation activities.
• Design and implement ETL/ELT solutions using Azure-native technologies.
• Ensure successful migration with minimal business disruption.

Client & Stakeholder Management

• Act as primary technology advisor for client stakeholders and executive leadership.
• Lead architecture discussions, workshops, and solution review sessions.
• Present solution roadmaps, risk assessments, and migration strategies to business and IT leadership.
• Manage client escalations, dependencies, and expectations throughout project lifecycle.

Team Leadership & Delivery Governance

• Allocate work across architects, engineers, analysts, and migration teams.
• Provide technical leadership and mentoring to delivery teams.
• Conduct solution reviews, code reviews, and architecture governance meetings.
• Monitor project scope, risks, dependencies, and delivery milestones.
• Ensure adherence to SDLC, Agile, and enterprise governance standards.

Data Governance & Security

• Establish data governance frameworks and metadata management strategies.
• Ensure compliance with HIPAA, PHI, PII, CMS, and healthcare regulatory requirements.
• Implement data security, encryption, data masking, and access control mechanisms.

Required Technical Skills

Azure Ecosystem

• Azure Databricks
• Azure Data Factory (ADF)
• Azure Synapse Analytics
• Azure Data Lake Storage Gen2 (ADLS)
• Azure SQL Database
• Azure Functions
• Azure Key Vault
• Azure Monitor
• Microsoft Fabric (preferred)

Data Engineering

• Apache Spark
• PySpark
• Delta Lake
• SQL
• Python
• ETL/ELT Design
• Data Modeling (Dimensional & Data Vault)
• Data Quality Frameworks

Enterprise Data Warehouse Experience

Experience with one or more:

• Teradata
• Netezza
• Oracle EDW
• SQL Server
• IBM DB2
• Hadoop Ecosystem

DevOps & Automation

• Azure DevOps
• CI/CD Pipelines
• Git
• Infrastructure as Code
• Deployment Automation

Healthcare Payer Expertise

Strong experience working with:

• Member Data
• Claims Data
• Provider Data
• Eligibility & Enrollment
• Benefits Configuration
• Encounter Data
• Risk Adjustment Data
• HEDIS Measures
• Pharmacy Data
• Value-Based Care Programs
• CMS Reporting Requirements

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or related field.
• 12 years of IT experience with at least:
  • 5 years in Azure Data & Analytics architecture.
  • 5 years in Healthcare Payer domain.
  • 3 large-scale cloud migration programs.
• Proven experience leading EDW to Azure Databricks migration initiatives.
• Strong stakeholder communication and executive presentation skills.
• Experience leading onsite-offshore delivery models.

Preferred Certifications

• Microsoft Certified: Azure Solutions Architect Expert
• Microsoft Certified: Azure Data Engineer Associate
• Databricks Certified Data Engineer Professional
• Databricks Certified Solution Architect
• TOGAF Certification
• PMP / SAFe Agilist

Key Competencies

Functional

• Healthcare Payer Business Processes
• Requirements Analysis
• Solution Consulting
• Stakeholder Management
• Business Process Transformation

Technical

• Cloud Architecture
• Data Platform Modernization
• Data Migration Strategy
• Azure Databricks Lakehouse
• Data Governance

Leadership

• Client Engagement
• Team Management
• Work Allocation & Resource Planning
• Delivery Governance
• Executive Communication
• Risk and Issue Management

Success Metrics

• Successful migration of EDW assets to Azure Databricks.
• Improved data platform scalability and performance.
• Reduced operational costs through cloud modernization.
• High client satisfaction and stakeholder engagement.
• Strong adherence to delivery timelines and quality standards.
• Effective governance and team productivity.