Azure Databricks Architect

Job Title: Azure Databricks Architect

Location: Seattle, WA (Hybrid 3 Days Onsite)
Duration:Full-Time

Job Summary

We are seeking an experienced Azure Databricks Architect to lead the design, implementation, and optimization of enterprise-scale data platforms on Azure. The ideal candidate will have deep expertise in Azure Databricks, Spark, Delta Lake, and modern Lakehouse architecture, with strong experience designing secure, scalable, and high-performance data solutions. Knowledge of Unity Catalog, Medallion Architecture, and Infrastructure as Code is highly preferred. These are all considered current best practices for enterprise Azure Databricks environments.

Required Skills

• 12 years of IT experience with 5 years as a Data/Cloud Architect.
• Strong hands-on experience with Azure Databricks.
• Expertise in PySpark, Spark SQL, Python, and SQL.
• Strong knowledge of Delta Lake, Lakehouse Architecture, and Medallion (Bronze/Silver/Gold) Architecture.
• Experience with Unity Catalog, data governance, RBAC, and data lineage.
• Experience designing scalable ETL/ELT pipelines.
• Hands-on experience with Azure Data Factory (ADF).
• Strong experience with Azure Data Lake Storage (ADLS Gen2).
• Experience with Azure Synapse Analytics or Microsoft Fabric is a plus.
• Experience with CI/CD using Azure DevOps, Git, and Infrastructure as Code (Terraform/Bicep/ARM).
• Strong understanding of performance tuning, cluster optimization, partitioning, caching, and cost optimization.
• Experience implementing data quality, monitoring, logging, and observability.
• Knowledge of streaming technologies (Structured Streaming/Event Hubs/Kafka) is preferred.
• Excellent communication and stakeholder management skills.

Responsibilities

• Design and architect enterprise-scale Azure Lakehouse solutions.
• Define end-to-end data architecture, ingestion, transformation, and serving layers.
• Lead Azure Databricks implementation using best practices.
• Design secure data governance using Unity Catalog and role-based access controls.
• Optimize Spark workloads for performance and cost.
• Implement reusable frameworks, coding standards, and CI/CD pipelines.
• Work closely with business, engineering, and cloud infrastructure teams.
• Provide architectural guidance, code reviews, and technical leadership.
• Ensure scalability, reliability, security, and disaster recovery across the platform.

Nice to Have

• Databricks Certified Professional/Architect certification.
• Azure Solutions Architect or Azure Data Engineer certification.
• Experience with MLflow, MLOps, or AI/GenAI workloads on Databricks.
• Experience with Power BI integration.