What are the responsibilities and job description for the Azure Databricks Architect with DBT (Only W2) position at QData Inc?
Role: Azure Databricks Architect with DBT
Industry: Healthcare
Location: Remote
Type: Contract – To – Hire
Job Description:
We are seeking an experienced Azure Databricks Architect with strong expertise in DBT (Data Build Tool) to design, architect, and implement modern data platforms on Azure. The ideal candidate will have deep hands-on experience in Databricks, data modeling, pipeline development, cloud architecture, and enterprise-grade data solutions.
This role will be responsible for leading end-to-end data architecture, designing scalable ELT/ETL frameworks, optimizing performance, and enabling data transformation using DBT and Spark.
Key Responsibilities
Architecture & Design
- Architect and design scalable data platforms using Azure Databricks, Delta Lake, Spark, and DBT
- Define data lakehouse architecture, governance, cataloging, and standards
- Develop end-to-end ELT/ETL pipelines using Databricks & DBT
- Create reusable frameworks for ingestion, transformation, and orchestration
Development & Implementation
- Build DBT models, macros, tests, and project structures
- Implement Delta Lake optimization techniques: Z-ordering, partitioning, caching
- Integrate Databricks with Azure Data Factory, Azure Data Lake, Synapse, Purview
- Develop CI/CD pipelines for Databricks & DBT using Azure DevOps
Performance & Optimization
- Optimize Spark jobs, cluster configurations, and resource utilization
- Troubleshoot performance bottlenecks and data quality issues
- Ensure secure, scalable, and cost-efficient platform design
Collaboration & Leadership
- Work closely with data engineers, analysts, architects, and business stakeholders
- Provide technical leadership, best practices, and code reviews
- Support production deployments and multi-environment management
Required Skills & Experience
- 8 years in data engineering / architecture
- 5 years hands-on experience with Azure Databricks & Spark
- 2 years experience with DBT (Core or Cloud)
- Strong understanding of Delta Lake, Medallion Architecture, and SQL
- Experience integrating Azure services: ADF, ADLS Gen2, Synapse, Key Vault
- Expertise in Python, PySpark, SQL, and data modeling
- Experience with Azure DevOps CI/CD (YAML pipelines preferred)
- Strong knowledge of data governance, security, lineage, and metadata management
- Excellent communication, design documentation, and client-facing skills