DataBricks Lead / Architect

At Staffworxs, we don’t just connect talent — we power transformation. Headquartered in Frisco, TX, with teams in Bengaluru and Hyderabad, we combine global reach with deep expertise. Our Digital & Data Analytics practice drives growth and innovation for some of the world’s top brands, who continue to retain us as their trusted partner. If you’re ready to make an impact, you’re in the right place.

Job Details:

Job Title: DataBricks Lead / Architect

Experience: 8–12 Years

Location: Dearborn, Michigan, United States (Weekly 2 day onsite)

Job Summary

We are looking for an experienced DataBricks Lead / Architect with deep expertise in Azure and Databricks to design and deliver scalable data platforms in a manufacturing environment. The ideal candidate combines strong hands-on engineering skills with architectural vision, governance practices, and cloud-native data solutions on Azure.

Key Responsibilities

• Design and implement scalable, high-performance data architectures using Databricks on Azure
• Lead the development of end-to-end data pipelines and lakehouse solutions
• Define and enforce data governance, security, and access control strategies
• Collaborate with business stakeholders, data engineers, and analysts to deliver reliable data solutions
• Drive best practices in data engineering, performance optimization, and cost efficiency
• Oversee CI/CD pipelines and DevOps processes for data platforms
• Provide technical leadership, mentoring, and guidance to engineering teams

Core Technical Skills

Databricks & Spark

• Advanced expertise in Apache Spark using PySpark/Scala for large-scale data processing
• Strong performance tuning skills (Spark UI, partitioning, joins, caching, shuffle optimization)
• Deep knowledge of the Databricks platform: notebooks, workflows/jobs, cluster configuration, and Databricks SQL
• Hands-on experience implementing Medallion Architecture (Bronze/Silver/Gold layers)

Delta Lake & Lakehouse

• Strong experience with Delta Lake including ACID transactions, schema enforcement/evolution, MERGE operations, and time travel
• Ability to design optimized Delta tables for scalable lakehouse solutions

Data Governance & Security

• Expertise in Unity Catalog including catalogs, schemas, permissions, lineage, and secure data sharing

Azure Ecosystem

• Strong experience with Microsoft Azure services including:
• Azure Data Lake Storage
• Azure Data Factory / Synapse Pipelines
• Event Hubs / IoT integrations
• Networking, IAM, Key Vault, and monitoring

Data Modeling & SQL

• Advanced SQL skills for transformations, aggregations, and analytics-ready datasets
• Strong understanding of dimensional modeling (star/snowflake schemas)

DevOps & CI/CD

• Experience with Git-based workflows and CI/CD pipelines
• Familiarity with Azure DevOps or GitHub Actions for deploying Databricks assets

Required Skills & Qualifications

• 8–12 years of experience in data engineering and architecture
• Proven expertise in Databricks and Azure-based data platforms
• Strong problem-solving and analytical skills
• Excellent communication and stakeholder management abilities
• Experience in manufacturing domain is highly preferred

Preferred Certifications

• Microsoft Certified: Azure Solutions Architect Expert
• Microsoft Certified: Azure Data Engineer Associate
• Databricks Certified Data Engineer (Associate/Professional)
• Databricks Hadoop Migration Architect (or equivalent)

Staffworxs is an Equal Opportunity Employer. We celebrate diversity and are committed to creating an inclusive workplace for all employees, regardless of race, color, religion, gender, sexual orientation, national origin, age, disability, or veteran status.