What are the responsibilities and job description for the Databricks Architect position at Inherent Technologies?
Role Overview
This role is part of a multi year enterprise initiative to modernize data platforms by migrating from legacy and on prem environments to cloud native, governed, and scalable architectures.
The role focuses on migrating enterprise data workloads to Databricks on strategic cloud platforms, enabling standardized data engineering, analytics, centralized reporting, reconciliation utilities, and AI/ML use cases-while adhering to controls, security, resilience, and regulatory compliance.
Key Responsibilities
Lead and execute migration of legacy data platforms (on prem / non standard tools) to Databricks on cloud under the Olympus program
Perform application, data, and pipeline refactoring to cloud native Databricks patterns
Drive migration planning including dependency analysis, sequencing, and cutover strategy
Support coexistence models and transition from dual run to cloud only execution
Databricks Lakehouse Engineering
Design and implement Databricks Lakehouse architecture (Bronze / Silver / Gold)
Build scalable batch and streaming pipelines using PySpark, Spark SQL
Leverage Delta Lake for reliability, versioning, and performance
Optimize compute usage and cost in line with enterprise cloud efficiency goals
Enterprise Data Controls & Governance
Embed data quality, reconciliation, and completeness controls as part of migration
Ensure migrated workloads meet EDO governance, MCA, and audit requirements
Maintain lineage, traceability, and explainability across migrated assets
Support risk critical use cases (Finance, Ops, Recon, Reporting)
Cloud Security & Resilience
Implement cloud aligned RBAC, identity controls, and secure access patterns
Enforce data encryption, masking, and classification standards
Ensure workloads meet operational resilience and recovery expectations
Partner with cloud platform and security teams for certification and sign off
Reporting, Analytics & AI Enablement
Enable downstream BI, regulatory reporting, and MI workloads on Databricks
Support centralized reporting programs (e.g., ARA, GRU related use cases)
Prepare data foundations for AI / ML and Agentic workflows post migration
Required Qualifications
8 12 years in data engineering / platform modernization
Strong hands on experience with Databricks in large scale enterprises
Proven experience delivering cloud migration programs (on prem cloud)
Deep expertise in Apache Spark, PySpark, Spark SQL
Experience embedding controls, reconciliation, and data quality in migrations
Preferred Qualifications
Experience in regulated environments (banking / financial services preferred)
Experience with Citi Olympus or equivalent enterprise cloud programs
Knowledge of legacy data platforms and modernization patterns
Familiarity with Finance, Ops, Recon, or Balance Sheet data domains
Exposure to MLflow, AI pipelines, or GenAI enablement on cloud
Strong understanding of run the bank vs change the bank execution
Behavioral & Delivery Expectations
Strong ownership and execution mindset
Comfortable operating in large, multi vendor transformation programs
Ability to engage with Technology, Operations, Risk, and Audit stakeholders
Disciplined approach to migration risk, controls, and documentation
This role is part of a multi year enterprise initiative to modernize data platforms by migrating from legacy and on prem environments to cloud native, governed, and scalable architectures.
The role focuses on migrating enterprise data workloads to Databricks on strategic cloud platforms, enabling standardized data engineering, analytics, centralized reporting, reconciliation utilities, and AI/ML use cases-while adhering to controls, security, resilience, and regulatory compliance.
Key Responsibilities
Lead and execute migration of legacy data platforms (on prem / non standard tools) to Databricks on cloud under the Olympus program
Perform application, data, and pipeline refactoring to cloud native Databricks patterns
Drive migration planning including dependency analysis, sequencing, and cutover strategy
Support coexistence models and transition from dual run to cloud only execution
Databricks Lakehouse Engineering
Design and implement Databricks Lakehouse architecture (Bronze / Silver / Gold)
Build scalable batch and streaming pipelines using PySpark, Spark SQL
Leverage Delta Lake for reliability, versioning, and performance
Optimize compute usage and cost in line with enterprise cloud efficiency goals
Enterprise Data Controls & Governance
Embed data quality, reconciliation, and completeness controls as part of migration
Ensure migrated workloads meet EDO governance, MCA, and audit requirements
Maintain lineage, traceability, and explainability across migrated assets
Support risk critical use cases (Finance, Ops, Recon, Reporting)
Cloud Security & Resilience
Implement cloud aligned RBAC, identity controls, and secure access patterns
Enforce data encryption, masking, and classification standards
Ensure workloads meet operational resilience and recovery expectations
Partner with cloud platform and security teams for certification and sign off
Reporting, Analytics & AI Enablement
Enable downstream BI, regulatory reporting, and MI workloads on Databricks
Support centralized reporting programs (e.g., ARA, GRU related use cases)
Prepare data foundations for AI / ML and Agentic workflows post migration
Required Qualifications
8 12 years in data engineering / platform modernization
Strong hands on experience with Databricks in large scale enterprises
Proven experience delivering cloud migration programs (on prem cloud)
Deep expertise in Apache Spark, PySpark, Spark SQL
Experience embedding controls, reconciliation, and data quality in migrations
Preferred Qualifications
Experience in regulated environments (banking / financial services preferred)
Experience with Citi Olympus or equivalent enterprise cloud programs
Knowledge of legacy data platforms and modernization patterns
Familiarity with Finance, Ops, Recon, or Balance Sheet data domains
Exposure to MLflow, AI pipelines, or GenAI enablement on cloud
Strong understanding of run the bank vs change the bank execution
Behavioral & Delivery Expectations
Strong ownership and execution mindset
Comfortable operating in large, multi vendor transformation programs
Ability to engage with Technology, Operations, Risk, and Audit stakeholders
Disciplined approach to migration risk, controls, and documentation