What are the responsibilities and job description for the Lead Data Engineer position at EXL?
About Us:
Our passion at EXL is enabling organizations to achieve a measurable, competitive advantage through Data Management. We partner with our clients to design and implement scalable, accessible, and innovative solutions. Our goal is to make sense of data to drive client’s business forward through data strategy, data governance and building cloud data platforms. EXL is a leading data analytics and digital operations and solutions company that partners with clients to improve business outcomes and unlock growth. By bringing together deep domain expertise with robust data, powerful analytics, cloud, artificial intelligence (“AI”) and machine learning (“ML”), we create agile, scalable solutions and execute complex operations for the world’s leading corporations in industries including insurance, healthcare, banking and financial services, media, and retail, among others. Focused on driving faster decision-making and transforming operating models, EXL was founded on the core values of innovation, collaboration, excellence, integrity and respect. Headquarters in New York, our team is over 44,500 strong, with more than 50 offices spanning six continents.
Position: Lead Data Engineer
Location: Los Angeles, CA
Exp Level: 9–12 years
Employment Type: Full-Time (Hybrid)
Role Overview
- Strong data engineering background
- Databricks is a must, along with experience in data modeling, building pipelines, orchestration, and supporting reporting teams
- Collaborate with client stakeholders to gather requirements, structure solutions, and ensure high‑quality, timely delivery.
- Experience working in the Databricks tech stack with strong proficiency in SQL, Python, and PySpark
- Design and optimize data models, data marts, and Lakehouse/warehouse layers with strong focus on medallion architecture, query optimization, and performance engineering.
- Build, orchestrate, and monitor scalable data pipelines on Databricks, ensuring reliable ingestion, transformation, CDC handling, and incremental load strategies.
- Manage end‑to‑end pipeline operations including performance tuning, data quality monitoring, alerting, and issue resolution across production workloads.
- Lead a project team of data engineers supporting multiple workstreams and provide technical leadership through code reviews, best‑practice guidance, reusable pattern creation, and mentorship to engineering team members.
- Prepare and maintain project documentation to support project execution and delivery.
Expected work split
50% Technical – Data modeling, hands-on coding, orchestration, and pipeline monitoring.
50% Management– Client Collaboration, requirements gathering, designing technical solutions, presentations, global team management and mentoring.
Qualifications (Required):
- 8-12 years’ experience in data engineering and analytics roles
- Bachelor’s or master’s degree in analytics, computer science/engineering, economics, mathematics, or related areas.
- Experience building and maintaining ETL/ELT pipelines
- Solid understanding of data warehousing concepts and dimensional data modeling
- Familiarity with workflow orchestration tools such as Airflow or similar
- Experience working with cloud data platforms or modern data infrastructure
- Entrepreneurial hands-on approach to work. Demonstrated leadership ability and willingness to take initiative
- Superior analytical and problem-solving skills
- Outstanding written and verbal communication skills
- Effective time management and attention to detail
- Hands-on experience in using SQL, Python and Workflow Schedulers (Apache Airflow, Cron)
- Experience in leading team and coordinating with internal / external stakeholders
Qualifications (Preferred):
- Experience in using Cloud Platforms (AWS / GCP / Azure)
- Experience in using Visualization tools (Tableau / Power BI)
- Experience with Big Data Technologies (Hadoop, Hive, Hbase, Pig, Spark, etc.)