What are the responsibilities and job description for the Azure Data Engineer position at Madison-Davis, LLC?
A leading enterprise organization is seeking an Azure Data Engineer to support the development and expansion of a modern cloud-based data platform. This role will focus on designing, building, and optimizing scalable data pipelines that support enterprise analytics, operational reporting, and data integration initiatives.
The ideal candidate is a hands-on data engineer who enjoys solving complex data challenges, working with modern cloud technologies, and collaborating with business and technical stakeholders to deliver reliable and scalable data solutions.
What You'll Tackle:
- Design, develop, and maintain scalable data pipelines using Azure-based technologies.
- Build and support ingestion frameworks for internal and external data sources.
- Develop and optimize batch and near real-time data processing workflows.
- Implement and maintain data lake and lakehouse architectures.
- Collaborate with business, engineering, architecture, and data teams to support enterprise data initiatives.
- Ensure data quality, integrity, and reliability across data pipelines and platforms.
- Optimize performance of large-scale data processing workloads.
- Support data integration, transformation, and orchestration activities.
- Participate in Agile development processes, sprint planning, and delivery cycles.
- Contribute to platform modernization and cloud transformation initiatives.
- Develop technical documentation, standards, and operational procedures.
What You Bring:
- 3–5 years of experience in data engineering, data integration, or related disciplines.
- Strong expertise in Python, PySpark, and SQL.
- Hands-on experience with Azure Databricks.
- Experience with Azure cloud services and modern data platforms.
- Experience developing data ingestion pipelines using APIs and external data sources.
- Strong understanding of data modeling, transformation, and pipeline optimization.
- Experience working with relational and/or NoSQL databases.
- Experience developing APIs using Python frameworks such as FastAPI or similar technologies.
- Strong analytical, troubleshooting, and problem-solving skills.
- Excellent communication and stakeholder collaboration abilities.
Nice to Have:
- Experience supporting enterprise-scale data lake or lakehouse platforms.
- Familiarity with Azure Synapse, Microsoft Fabric, or comparable analytics platforms.
- Experience working with large-scale cloud data environments.
- Exposure to governance, security, or compliance-related data initiatives.
- Azure certifications or related cloud credentials.
What Success Looks Like:
- Reliable delivery of scalable, high-performance data pipelines.
- Strong data quality, integrity, and operational stability.
- Successful collaboration across business and technical teams.
- Continuous improvement of cloud data platform capabilities.
- Contribution to enterprise modernization and data transformation initiatives.
- Delivery of efficient, well-documented, and maintainable data solutions.