What are the responsibilities and job description for the Data Engineer position at AP Professionals?
Seeking an experienced Data Engineer to support and extend existing data ingestion and pipeline frameworks within a modern data platform environment. This role focuses on operating and enhancing established systems rather than redesigning or re‑architecting the platform.
The Data Engineer will work as part of an internal scrum team and receive work primarily from platform leads and architects. The role emphasizes distributed data processing, production‑grade pipelines, and software engineering best practices.
Key Responsibilities
- Operate and extend an existing metadata‑driven ingestion framework (MDIF)
- Support new data source discovery and onboarding
- Build and maintain scalable data pipelines using Databricks and Python
- Prepare data for analytics, reporting, and future AI/ML use cases
- Collaborate within an internal agile scrum team following established frameworks
- Follow software development lifecycle practices, including version control and CI/CD
Required Technical Skills
- Databricks (critical)
- Python
- Modern data engineering and big data experience with distributed data processing
- GitHub experience, including:
- Feature branching
- CI/CD pipelines
- Software development lifecycle practices
Cloud & Platform Environment
- AWS is the underlying cloud infrastructure
- Databricks expertise is prioritized over general cloud specialization
Healthcare Domain Required Experience
- Payer‑side healthcare experience
- Familiarity with one or more of the following:
- Claims data
- Medicare Advantage and Medicaid
- ACO and value‑based care models
- STAR ratings and quality metrics
- Regulatory and compliance‑driven reporting
Overview
- Role is generally not customer‑facing
- Some interaction may occur for senior contributors or lead‑level responsibilities
- This is not an entry‑level position
- This is not an over‑engineered architect role
- Ideal candidates are hands‑on practitioners who can be productive quickly within established platforms