Databricks Lead / Soln. Architect

Job Title:

Databricks Leader / Solution Architect

Location: Minneapolis, MN – Hybrid

Job Description:

We are seeking an experienced DBX Leader / Solution Architect with 12–15 years of IT experience and strong expertise in Microsoft Azure and Databricks. The ideal candidate will have strong solution architecture, data engineering, cloud architecture, and technical leadership experience.

Key Responsibilities:

• Design and architect enterprise-grade Azure cloud and data solutions.
• Design Databricks Lakehouse solutions using Delta Lake, Unity Catalog, Spark, and MLflow.
• Architect scalable ETL/ELT and real-time streaming solutions.
• Develop HLDs, LLDs, data flow diagrams, integration patterns, and security architectures.
• Lead architecture and solution design discussions with technical and business stakeholders.
• Guide data engineering, DevOps, analytics, and platform teams.
• Establish best practices for data governance, CI/CD, automation, security, and environment management.
• Conduct architecture assessments and recommend improvements for scalability, performance, resilience, and cost optimization.
• Participate in client workshops and executive-level technical discussions.

Required Skills:

• 12–15 years of overall IT experience.
• 5 years of Azure architecture experience.
• 5 years of Databricks experience.
• Strong expertise in Databricks, Spark, Delta Lake, Unity Catalog, and MLflow.
• Strong Python, SQL, and PySpark skills.
• Experience with Azure Data Factory, ADLS, Synapse, Azure Functions, Event Hubs, and Azure VMs.
• Strong understanding of data modeling, data governance, and metadata management.
• Experience with CI/CD, Terraform, Azure Bicep/ARM Templates, and GitOps.
• Strong knowledge of Azure VNets, NSGs, Private Endpoints, Azure AD, RBAC, encryption, and identity governance.
• Knowledge of security frameworks such as NIST and CIS benchmarks.
• Strong communication, stakeholder management, and architectural leadership skills.