What are the responsibilities and job description for the Urgent Hiring for the Position of Databricks Subject Matter Expert || Dallas TX position at Buzzclan?
BuzzClan is an elite business consulting firm collaborating to provide software, advisory & implementation services. BuzzClan is a certified partner for most of the tier 1 cloud, hardware & software providers. Being a vertically integrated solutions company, BuzzClan is known for their capability in the IT Services space.
Databricks SME – Lead / Principal
Location: Dallas, TX (Onsite, 5 days/week)
Role Overview
Seeking a highly experienced Databricks Subject Matter Expert to lead data platform architecture, engineering, and transformation initiatives. The role involves designing scalable Lakehouse solutions, advising senior stakeholders, leading technical teams, and supporting consulting and pre-sales activities.
Key Responsibilities
- Design enterprise-scale Databricks Lakehouse architectures using Delta Lake, Unity Catalog, and medallion architecture.
- Lead large-scale ETL/ELT pipelines using Apache Spark, PySpark/Scala, Delta Live Tables, and Databricks Workflows.
- Establish data governance, security, quality, lineage, and performance optimization standards.
- Guide ML workflows and MLOps implementation using MLflow, Feature Store, and Model Serving.
- Advise clients on data strategy, architecture, cloud adoption, and technical roadmaps.
- Lead architecture reviews, technical assessments, proposals, and client workshops.
- Mentor engineering teams and contribute to best practices and reusable frameworks.
Required Qualifications
- 10 years of experience in data engineering, analytics, or platform architecture.
- 5 years of hands-on Databricks experience with enterprise production deployments.
- Expert knowledge of Apache Spark, PySpark/Scala, Delta Lake, and SQL.
- Experience with cloud-based Lakehouse architectures on Azure, AWS, or GCP.
- Strong expertise in Unity Catalog, data governance, and lineage.
- Proven client-facing leadership and stakeholder communication skills.
- Consulting or professional services experience managing multiple engagements.
- Bachelor's degree in a relevant technical field or equivalent practical experience.
Preferred Qualifications
- Databricks Certified Data Engineer Professional or Machine Learning Professional.
- Experience with Delta Live Tables, Databricks Workflows, and Photon optimization.
- Knowledge of dbt, Terraform/Pulumi, and CI/CD pipelines.
- Production experience with MLflow, Feature Store, and Model Serving.
- Experience in Databricks consulting or partner practices.