Lead Data Engineer - Data Ingestion

Job Title: Lead Data Engineer

Position Summary:

The Lead Data Engineer serves as both a technical authority and individual contributor responsible for the delivery and continuous improvement of healthcare data products and platform components. This position requires a proven track record of architecting secure, high-throughput data pipelines and leading high-performing teams of 5–8 engineers.

The Lead Data Engineer translates business strategy into data engineering execution, guiding the team in the adoption of tools such as AWS Glue, Iceberg, SageMaker, and feature flag frameworks, while ensuring HIPAA and HITRUST compliance. Acting as a bridge between engineering, data science, and product functions, this role supports alignment during sprint planning and delivery, emphasizing clear communication, documentation, and team empowerment.

Essential Duties and Responsibilities:

• Architect and lead the adoption of serverless data platforms integrating AWS services such as Glue, Lambda, DynamoDB, and Lakehouse Formation.
• Lead GitOps strategy for infrastructure and pipeline deployment, ensuring compliance, rollback safety, and peer-reviewed workflows.
• Establish and enforce best practices for feature flag toggling strategies in critical data pipelines.
• Drive internal knowledge sharing and technical documentation standards across the engineering organization.
• Mentor and guide junior engineers to promote professional growth, collaboration, and best practices.
• Design scalable architectural solutions and effectively communicate complex technical concepts to both technical and non-technical audiences.
• Own technical environment and tooling strategy across the data engineering organization, ensuring standards, documentation, and collaboration practices are consistently applied.
• Maintain and oversee governance using:
• GitHub (repository standards and peer review)
• AWS Console (Glue, Iceberg, EMR, IAM governance)
• Jupyter/Python IDE (advanced notebook-based development)
• Lucidchart (platform-wide architecture diagrams)
• Microsoft Teams/Zoom (engineering leadership and stakeholder communications)
• Jira & Confluence (governance documentation and delivery tracking)

Knowledge, Skills, and Abilities:

• Excellent written, verbal, and interpersonal communication skills.
• Strong organizational skills and attention to detail.
• Excellent documentation, mentoring, and leadership abilities.
• Proven ability to motivate and lead teams, setting clear expectations and driving accountability.
• Ability to manage multiple concurrent projects and priorities in a dynamic environment.
• Skilled at communicating complex technical topics clearly and effectively to various audiences.
• Strong problem-solving and analytical thinking skills.
• Ability to operate effectively in collaborative, cross-functional environments.

Technical Skills:

• Programming Languages: Python, PySpark, Scala (10 years) – Directed large-scale data platform development with an emphasis on modularity and performance.
• Orchestration Tools: Airflow (6 years) – Defined standards for DAG design, observability, and reusability across teams.
• Cloud Platforms: AWS (Glue, EMR, Lambda, Step Functions) (5 years) – Designed hybrid orchestration strategies using serverless and EMR-based pipelines.
• Data Architecture: Iceberg (4–5 years) – Led lakehouse strategy including governance, versioning, and cost optimization.
• Architecture Leadership: Deep expertise in designing and optimizing data lake and lakehouse architectures at scale, including governance, schema evolution, cost efficiency, and access control.
• Containerization: Docker/Kubernetes (6–8 years) – Managed Kubernetes-based orchestration for enterprise data pipelines and established Docker standards aligned with compliance and DevOps strategies.
• Team Leadership: Directed a data engineering team of 5–8 engineers, driving architectural decisions, enforcing engineering standards, and championing Agile delivery practices within a high-performance environment.

Education and Experience Requirements:

• Bachelor’s degree in computer science, Engineering, or a related field, or equivalent combination of education and experience.
• 10 years of experience in data engineering roles.
• Extensive experience with large-scale data systems and cloud-based architectures (AWS required).
• Prior experience with healthcare or regulated datasets strongly preferred.

Compensation and Benefits:

The expected base salary for this position ranges from $125,000 – $150,000 USD with a bonus target of up to 5% of the base salary. While it is not typical to offer candidates at or near the top of the range, we do consider a wide range of factors when making offer decisions, including (but not limited to) the scope and responsibilities of the position, a candidate’s relevant skills, training, experience, education and where applicable, licensure and certifications obtained. We also consider organizational and market factors when making offer decisions. This range is a general guideline and represents a good faith estimate of what we would reasonably expect to pay for this position. We reserve the right to modify this range in the future as organizational and market conditions dictate.

At Convey, we believe that the right benefits package can improve the quality of our employees and their family members’ lives. Therefore, we offer a competitive total rewards package, which includes health and welfare benefits, 401(K) savings with employer matching, paid time off, life, AD&D and disability insurances, and various ancillary benefits, such as pet insurance, virtual physical therapy, and employee assistance programs.

Equal Employment Opportunity Statement

Convey Health Solutions is an Equal Opportunity Employer committed to fostering an inclusive and diverse workforce. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age, disability, genetic information, veteran status, or any other characteristic protected by applicable federal, state, or local laws.

Convey Health Solutions also provides reasonable accommodations to qualified individuals with disabilities in accordance with applicable laws. Applicants requiring accommodation during the application or interview process should contact the Human Resources department.