What are the responsibilities and job description for the Data Team Lead position at Henderson Scott?
Databricks Lead – Raleigh, NC (Hybrid)
We are seeking a senior Databricks Lead to support a large-scale enterprise data transformation initiative in Raleigh, North Carolina. This is a strong opportunity for an experienced data engineering professional who thrives in modern cloud environments and enjoys building scalable, high-performance data platforms.
Compensation is highly competitive with opportunities reaching $140K depending on experience.
Responsibilities
- Design, develop, and optimize ETL/ELT pipelines using Databricks, Apache Spark, and PySpark
- Build and tune complex SQL queries for enterprise-scale data transformation and analytics
- Develop Delta Lake architectures and scalable data workflows
- Collaborate with architects, analysts, and business stakeholders to deliver data-driven solutions
- Implement orchestration, monitoring, and CI/CD processes for Databricks workflows
- Ensure data quality, reliability, scalability, and production performance
- Support cloud-based data engineering initiatives within Azure environments
Required Experience
- 10 years of experience in data engineering or related fields
- 3–5 years of hands-on Databricks and Spark experience
- Strong SQL development and performance tuning expertise
- Experience with ETL processes and data modeling
- Python or Scala programming experience
- Strong communication and problem-solving skills
Preferred Experience
- Insurance industry experience strongly preferred
- Power BI exposure is a plus
Location
Raleigh, North Carolina (Hybrid)
Salary : $140,000