What are the responsibilities and job description for the Data Engineer position at InstantServe LLC?
We are seeking an experienced Data Engineer & Insights Analyst to support the Pennsylvania Longitudinal Data System (PALDS). The candidate will design and maintain scalable data pipelines, develop AWS-based data solutions, integrate complex datasets, ensure data quality, and build analytics solutions to support research, reporting, and data-driven decision-making.
Key Responsibilities
- Design, develop, and optimize scalable data pipelines and ETL processes.
- Build cloud-based solutions using AWS services such as S3, Glue, Redshift, EMR, Athena, and Lambda.
- Develop data workflows using Airflow, AWS Step Functions, or similar tools.
- Integrate, transform, and analyze large datasets using SQL and Python.
- Develop dashboards, reports, and analytical models.
- Implement data quality, governance, monitoring, and validation frameworks.
- Collaborate with data scientists, analysts, and stakeholders.
- Support data migration, modernization, and system integration initiatives.
Required Skills
- 5 years of experience with AWS cloud data services.
- 5 years of programming experience with Python, R, Java, Scala, or C .
- 5 years of SQL and relational/NoSQL database experience.
- 5 years of experience with Git, CI/CD, and data modeling.
- Experience with Hadoop, Spark, Kafka, and Airflow.
- Knowledge of data governance, metadata management, and data quality.
- Experience with REST APIs and integration tools such as Informatica, MuleSoft, Talend, or TIBCO.
- Strong analytical, problem-solving, and communication skills.
- Required: Experience working with student-level higher education data and linking it with employment and wage datasets.
Work Arrangement & Interview
- 100% Remote.
- Two interview rounds.
- First round via Microsoft Teams.
- Final-round interview in person in Harrisburg, PA.