What are the responsibilities and job description for the Lead Databricks Engineer / W2 position/12+yrs experience position at VentureSoft Global?
Lead Data Engineer – Databricks Data Modeling & Architecture
Location: Detroit, MI (Onsite)
Work Arrangement: 100% Onsite
Job Summary
We are seeking an experienced Databricks Data Engineer for a 6-month onsite contract engagement in Detroit, MI. The ideal candidate will design and maintain enterprise-grade data models and architectures across data warehouses, data lakes, and lakehouse platforms. This role combines hands-on data modeling expertise with architectural leadership, partnering with business and technology teams to deliver scalable, governed, and high-performance data solutions on Databricks and Microsoft Azure.
Key Responsibilities
Design Enterprise Data Models Develop and maintain conceptual, logical, and physical data models for data warehouses, data lakes, and lakehouse architectures using Databricks and modern data platforms.
Architect Scalable Data Solutions Design end-to-end data architectures leveraging Databricks, Delta Lake, Spark, and cloud platforms to support AI/ML and business intelligence initiatives.
Establish Data Governance & Standards Define data modeling standards, metadata management, data quality frameworks, and master data management governance processes to ensure trusted and compliant data assets.
Lead Data Integration & Optimization Architect and optimize batch, streaming, CDC, and ELT/ETL pipelines, ensuring high performance, scalability, efficiency, and reliability across enterprise data ecosystems.
Collaborate with Business & Technology Teams Partner with business stakeholders, data engineers, analysts, and AI/ML teams to translate requirements into scalable data products, reusable data domains, and enterprise-wide data solutions.
Required Skills & Qualifications
- Databricks Lakehouse Architecture – proven hands-on design and implementation experience
- Delta Lake, Unity Catalog, Lakeflow, Auto Loader
- PySpark and SQL – strong development and optimization skills
- Data Modeling – 3NF, Dimensional (Star/Snowflake), and Data Vault methodologies
- Data Governance, Data Quality (DQ), and Metadata Management
- Cloud Platform: Microsoft Azure (Azure Data Lake Storage, Azure Databricks, and related services)
- Experience designing and optimizing batch, streaming, CDC, and ELT/ETL data pipelines
- Strong understanding of data warehouse, data lake, and lakehouse design patterns
- Excellent communication skills with the ability to engage both technical teams and business stakeholders
Preferred Qualifications
- Databricks certification (e.g., Databricks Certified Data Engineer Professional or Data Architect)
- Microsoft Azure certification (e.g., Azure Data Engineer Associate / Solutions Architect)
- Experience supporting AI/ML and BI workloads on a lakehouse platform
- Prior experience in an enterprise or large-scale data modernization program
- Background in regulated or enterprise environments with formal governance requirements