What are the responsibilities and job description for the Principal Data Engineer position at Agility Partners?
We’re looking for a Lead Databricks Consultant to drive data platform strategy and delivery across enterprise environments. This role blends client-facing solutioning with hands-on architecture and technical leadership, helping organizations modernize data, analytics, and AI platforms using Databricks and cloud technologies.
What You’ll Do
- Lead discovery sessions to uncover business goals and define high-impact data use cases
- Design scalable Lakehouse architectures using Databricks (Delta Lake, Unity Catalog)
- Own presales efforts including solution design, proposals, and delivery roadmaps
- Guide engineering teams building batch and streaming pipelines with Spark, Python, and SQL
- Establish best practices across CI/CD, data quality, observability, and governance
- Deliver presentations, demos, and whiteboarding sessions to business and technical stakeholders
- Drive performance optimization, cost efficiency, and platform adoption
What You Bring
- Experience leading client-facing data or analytics engagements
- Strong Databricks expertise (Delta Lake, Workflows/Jobs, Databricks SQL)
- Hands-on skills with Python, PySpark, and SQL
- Cloud experience (Azure, AWS, or GCP)
- Ability to translate business requirements into scalable technical solutions
- Familiarity with data governance, security, and access controls
Nice to Have
- Experience with large-scale data platform modernization or migration efforts
- Background in consulting or professional services
Why Join
- Lead high-impact data and AI initiatives
- Work in a fast-paced, collaborative consulting environment
- Exposure to modern cloud and data technologies
Salary : $220,000 - $250,000