What are the responsibilities and job description for the Data Engineer position at Agility Partners?
Agility Partners is seeking a qualified Engineer to fill an open position with one of our clients in the energy industry. This is an exciting opportunity to build and enhance cloud-based data platforms, backend services, and AI-enabled applications that support data-driven business solutions. The ideal candidate will combine data engineering and software development expertise to create reliable data pipelines, scalable APIs, and integrated analytics capabilities using modern Azure technologies.
A Little About This Gig
- Design, develop, and maintain scalable data pipelines and orchestration workflows using Azure Data Factory, Azure Synapse, Azure Data Lake Storage Gen2, SQL, and Python.
- Build and support backend services, REST APIs, and application integrations using Python, FastAPI, Azure Functions, and containerized solutions.
- Implement data quality checks, validation frameworks, automated testing, and CI/CD processes to improve reliability and streamline deployment.
- Integrate machine learning and AI capabilities into data platforms and applications, including feature pipelines, model-serving APIs, and agentic architectures.
The Ideal Candidate
- Hands-on experience with data engineering tools and technologies, including Azure Data Factory, Azure Synapse, Azure Data Lake Storage Gen2, SQL, and Python.
- Experience developing and maintaining data pipelines, orchestration workflows, and data integration solutions.
- Proficiency in Python-based software development, including building backend services and REST APIs with frameworks such as FastAPI.
- Familiarity with Azure Functions, containerized services, and cloud-based application architecture.
- Experience with GitHub, CI/CD pipelines, automated testing, and software deployment best practices.
- Understanding of asynchronous or event-driven architectures and integrating applications with data platforms.