What are the responsibilities and job description for the Azure Data Engineer position at STEM Search Group?
What you'll do...
- Help to architect and scale enterprise cloud data warehouses to deliver immediate data accessibility across business units.
- Design, construct, and automate robust data pipelines for large-scale data ingestion, cleansing, and cross-platform integration.
- Transform raw data science and AI prototypes into production-ready, highly reliable machine learning models.
- Maximize system efficiency by performance-tuning data integration pipelines and analytical queries.
- Build custom, automated health-monitoring and anomaly-detection tools to safeguard pipeline integrity and enterprise data quality.
What we're looking for...
- Extensive experience designing modern data lakehouses and schemas utilizing the Microsoft SQL Server ecosystem.
- Advanced proficiency orchestrating enterprise data movement and workflows using Azure Data Factory.
- Hands-on expertise using Databricks and Spark to process, manipulate, and optimize massive, complex datasets.
- Strong background writing clean, production-grade Python or modern scripting languages to handle automation and data transformations.
- Deep understanding of cloud storage architectures, relational database mechanics, and continuous integration/continuous deployment (CI/CD) pipelines.