Our client is seeking a talented and experienced Azure Data Engineer to join our team. The ideal candidate will have a strong background in data engineering, Azure Data Factory (ADF), and Azure Synapse, along with experience ingesting and consolidating data from various data sources including multiple ERP systems. This role requires expertise in designing and implementing data pipelines, data transformation, and data warehousing solutions using Azure services. The Senior Azure Data Engineer will play a crucial role in ensuring the security, reliability, and performance of our data infrastructure while collaborating with cross-functional teams to meet business objectives.

Responsibilities:

• Collaborate with stakeholders to understand data requirements and design and maintain secure and compliant data processing pipelines using Azure Data Factory, Azure Synapse, and other tools.
• Ingest data from various sources such as databases, files, APIs, and streaming platforms into Azure, ensuring efficient and reliable data transfer.
• Perform data transformation and processing tasks using Azure Synapse to prepare data for analysis and reporting.
• Select and manage appropriate data storage solutions in Azure, including Azure SQL Database, Azure Data Lake Storage, and Azure Blob Storage, based on data requirements.
• Successful data movements and transformations using various file formats such as Delta, Parquet, Avro, and CSV.
• Implement and optimize data warehousing solutions using Azure Synapse SQL and Azure Synapse Analytics to support large-scale analytics and reporting.
• Collaborate with data scientists and analysts to design and implement data models for analytical workloads using Azure Analysis Services or Power BI.
• Implement DevOps and automation practices to automate data pipelines, monitor data processes, and manage deployments using Azure DevOps, Azure Monitor, and Azure Automation.
• Support and troubleshoot on-premises gateways.

Requirements:

• Minimum of bachelor s degree in computer science, Engineering, Mathematics, or related fields
• 5 years of work experience in designing, developing, and troubleshooting complex SQL queries, on DBMS solutions such as Snowflake, Oracle, MS SQL Server, PostgreSQL, and MySQL
• 3 years of hands-on experience in ETL/ELT tools
• 3 years of work experience with Data Platforms
• 1 year of development experience in Python or similar scripting languages for data automation
• Proven experience as an Azure Data Engineer or similar role, with a focus on designing and implementing data pipelines and data warehousing solutions using Azure services.
• Expertise in Azure Data Factory (ADF), Azure Synapse, and multiple ERP systems.
• Experience working with On-Premises and Cloud systems and data sources.
• Strong analytical skills in working with unstructured datasets and designing efficient data processing workflows.
• Proficiency in coding and scripting languages such as SQL, Python, or PowerShell.
• Implement DevOps and automation practices to automate data pipelines, monitor data processes, and manage deployments using Azure DevOps, Azure Monitor, and Azure Automation.
• Excellent communication skills, with the ability to collaborate effectively with cross-functional teams and drive alignment towards common goals.
• Regularly check and maintain data quality by validating data against predefined standards and governance policies.
• Fix data quality issues and errors in data pipelines to maintain accurate data.

Nice to Have:

• Certifications such as Azure Solutions Architect Expert or Azure DevOps Engineer Expert.
• Experience with other Azure services and frameworks beyond ADF and Synapse including Azure Databricks and Snowflake.
• Knowledge of data modeling and analysis techniques using Azure services like Azure Analysis Services.
• Experience with Microsoft Fabric
• Experience with Informatica Data Quality and MDM